Graphite’s Post

Transcript

So at Graphite, we want to use AI to help our users create more impactful content. So to say a little bit more about that, you know, there's been a lot of interest in using AI for content creation, including a lot of interest in using generative AI to essentially fully auto generate content. So how do we think about that at Graphite? Well, like Graphite, we're very concerned with outcomes. What we care about is creating good outcomes for our customers and we think that given the current state of the technology, the best way to create high quality, impactful content right now is through AI assistance. Rather than full automation. So that's been our focus and that is the way that we're thinking about it currently. So the reason that there's been a lot of excitement about AI and content creation recently has been there have been these large advances maybe over the last ten years or so. And in particular, these advances were really powered by scaling up neural networks, so including more parameters in the model, using larger training datasets, as well as the development of some new architectures like Transformers. But the summary of all of those improvements is that essentially AI or machines more generally are able to understand text much more deeply. Than they ever have before. And a way to understand how these models interpret text is through embeddings. So deep neural networks are able to learn these compact latent features that are called embeddings that essentially encode mathematically the meaning of the text. So you take some text, you put it through a model, you get out an embedding and that embedding encodes its meaning. And more specifically what it embedding is, is is a vector. It's. Essentially a list of numbers and those lists of numbers can be interpreted as points and we can plot them. So here we've plotted some embeddings of keywords, so the axis of this graph don't really matter. But what's important is that keywords that appear close to each other in this plot have similar meaning or semantically similar to each other. So in our toy example here, you'll see that keywords that are related and relevant to the demo that Marcos gave for 4th wall How to make merch on YouTube websites. Sell merchandise and custom plushies are all close to each other. They sort of cluster together in this plot, and then there's a second cluster of transformer model and what is an LLM? So obviously this is a group of keywords that's about. AI and Machine learning. And they are grouped together and far apart from the 4th wall keywords. But the summary is that machines understand text much more deeply than they ever have before, and search engines are making heavy use of these advances. So if you think about what does the search engine try to do, essentially a lot of what a search engine is doing is trying to interpret text. They want to look at keywords or searches. They want to understand what the user's intent was. They want to understand what the user's questions. What questions does the user have. What type of content are they looking for? Are these keywords similar to each other? And then similarly on pages of documents? So they want to be able to, you know, they crawl the entire web and they want to be able to say that these documents are related to each other. What's the topical structure of this document? What questions does this document answer? Those sorts of things? And then of course, you have the matching part. So does this document satisfy the intent for this keyword? So over the last, let's say, five years or so, Google has been rolling out more and more. Improvements to search based on this idea that we can understand text much better than ever before. And to be clear, searches that have been trying to solve these problems for a long time. I mean, Google has been working on these problems for years, but the difference is that the technology now makes all of the solutions to these problems much, much better. They all work much better than they than they ever have. So when we started working in this area I graphite what we were thinking was, well, why don't we use the exact same types of technology? Neural networks, embeddings, ranking models, relevance. Models. Why don't we use basically the same kind of technology that search engines are now using to power content creation? And so we want to solve problems like automating tedious parts of the research. When creating content, we wanna surface insights from large datasets, especially insights that are about search intent. So we wanna really deeply understand search intent. We want to assist users in the writing process, give them guidance about what they should include, and with everything we do. We want to optimize for outcomes and have our algorithms improve with feedback loops. Let's go into a few specific examples that kind of highlight maybe a peek under the hood at what's actually happening for some of the examples that Marco showed. So first, let's talk about topics. So recall that topics are groups of semantically related keywords that share the same search intent. And so if we want to understand whether some keywords share the same search intent, we have to have some way of comparing them. So the question is, how are we going to compare them? Well, there are two interesting details about the way that we compare them. The 1st is that rather than comparing the raw keywords, we actually want to compare Serps. So part of this motivation for this is we want to optimize for outcomes, right. So we want to have similarity metrics that are essentially consistent with the channel that we're optimizing for. Secondarily, you know, as I mentioned, what search engines are really trying to do is to understand search intent. So search engine has taken the keyword and it's crawled the entire Internet and it. Has mountains of user data and advanced algorithms, and using all of that information, they decide that what appears on this search engine result page is the best answer to the user's query. And so it makes sense for us to try to piggyback on top of that when we're computing similarities. But then additionally, once we have these Serbs, we're embedding them, so we're running them through neural network models to get embeddings, and this helps to make the similarities more accurate. So let's look at the plot now. So in this plot, this is real data. And each point here represents A keyword. And what we've done is we've taken three seed keywords, how to make merch on YouTube in green, how to make merch in orange, and custom plush toys in blue. And we've created topics around them. So we basically expanded them out to their neighboring keywords that we think have the same search intent. So an interesting observation from looking at this plot is that if you just thought about how to make merge and how to make merch on YouTube, you might say that you know there's high string. Overlap here. So these are really the same intent and we can write one article that includes a section about specifically about YouTube, but we'd have one how to make merge article. But what this the data actually shows is that these are separate topics, they tend to have different results and we should actually make two different pages for them. Second thing to note with the topic, custom plush toys, there are many different ways that you might refer to a plush toy. So there's plushies, plush toys, stuffed animals, stuffed toys. I think there might have been another one in there as well. And what we're able to do is to group all of those keywords together into one topic. And so not only are we able to kind of target that entire cluster, but then we can also provide guidance related to that. Specifically that because users ask for this content in a few different ways, we should actually talk about plushies in a few different ways. In the article that we wrote. Second example, let's talk about topical authority. So topical authority. I think is a concept that people are increasingly familiar with and it's importance is becoming increasingly well known that it's an important ranking factor in addition to just domain authority, as Marcos explained. But how do we actually measure it? How do we know whether? A particular domain like 4th wall has topical authority in a particular area. If we think about this as a machine learning problem, we can basically use the same SURF embeddings that we. We looked at in the previous slides about topics, and so here's what we're going to do. First, we'll plot all the 4th wall keywords. Those are the keywords in purple here, so these are all the keywords that 4th wall already ranks for. Then we can plot. Two more groups of keywords. One is the topic how to make merch on YouTube, the one we saw in the previous slide. The other one is a new topic, membership platforms. OK, so we've plotted those now and then what we can do? Is we can say. Let's look at the area around the topic, how to make merch on YouTube and see how many. Purple 4th Wall Keywords we find in that area, and here we'll actually see that this topic is kind of surrounded by fourth Law keywords, and it may be hard to see, but there are actually a few purple ones kind of mixed in amongst the green. So here because the coverage of this region around the topic is pretty high with 4th Wall Keywords, we would say that the topical Authority is high in our algorithm assigns an 85 and it's a score from zero to 100. OK, let's do the same with our other topic, membership platforms. So we do the same thing. We expand out around it a little bit and we say how many 4th Wall Keywords we see here. Well, now we see many fewer. There were a few right to the left of the topic, but none above it or on the sides. And you can generally see that this topic is kind of off to the side, away from the kind of center of mass of the 4th Wall Keywords. And so we would assign a fairly low tea here of 10. And as Marcos noted, we created this algorithm. But then we've actually also validated it by using it as a prioritization for what content to create and we've done a study and shown that if you create content that. Has high topical authority. On average it will rank more quickly and you can see that on our website. Next example is content scoring. So recall that this is the problem where we have a draft of the content and we want to figure out does the content is it doing a good job at covering the target topic or not. So two things that we do here that are kind of interesting, One is like some other tools, we're going to compare your content to competitors. So we're going to look and see who else is ranking and we're gonna look at that content and say, OK, well, if many other competitors are covering this concept, then you should probably cover the concept too. One difference there is that we're actually doing this for an entire topic rather than an individual keyword. So as Marco showed, sometimes often actually topics have hundreds of keywords and we're looking across hundreds of keywords and finding all of the. URLs competitors that rank for those keywords, and we're using those to inform this comparison to your competitors. The second thing is getting back to the point of trying to use the same technology you've got. Search engines use, we actually use. Transformer models here to evaluate the topical relevance of your content to the topic, keywords and. So this, this type of model is something that actually Google uses when determining relevance and it's trained on real ranking data. So in this example here we have an excerpt of an article that 4th Wall wrote and we have kind of a toy topic here so it's easy to visualize of two keywords, it's how to make merchandise and sell merchandise online. What we've done in the text here is we've highlighted the parts of the text that our model thinks are most relevant to these two keywords. So if you do that, you'll see that. The results make a lot of sense. So this section about how to make your own merch. 6 steps. This is considered to be very relevant for how to make merchandise. So creating your own merch. Here are 6 steps to help you on your merch making journey, designing your merge, these kinds of things right these are all relevant to how to make merchandise. And then the second section is more relevant to how to sell merchandise online. So how to start selling your merch? The next step is to start selling. Start selling your merch. But it's not all about selling. So also this part about identify your products and target audience. Ends up being considered very relevant to this sell merchandise online. But what we can see from this is if we were only targeting 1 keyword, let's suppose we only targeted how to make merchandise, then we might not know that we need to create this how to start selling. Section, whereas in the graphic tool if you started to write content and you did not include this section, the overall content score would be low because you essentially wouldn't be addressing the intent of this sell merchandise online keyword. OK. Final example for me is my team is always working on doing R&D and we're trying to understand how to further improve the content creation process and sort of find interesting new problems to solve and one that we notice that really. Doesn't get addressed very much as far as I know in other tools and in the SEO community is that very often the search intent for a keyword is ambiguous. So let's suppose that we're 4th wall and we want to write about merch sellers. And so we might say, OK, my keyword is merch seller. Now let's create a topic around that and I'm going to write something about it. But what we can see here here we're plotting the topic actually as a graph, but the details don't particularly matter. But what's very obvious is that there are sort of two very densely connected. Sub clusters connected to Marcella with merch seller in the middle kind of joining them. So the cluster on the lower right is basically the types of keywords that you would probably expect if you typed merge seller into tool. It's about merchandise stores and different types of merch sellers and that sort of thing. The second cluster on the upper left is something that I didn't know about until I created this example. So there's a product from Amazon called Merch on Demand where they will make your merge. And for all of the keywords on the upper left cluster. When they include the word merch, they're actually referring to this Amazon Merch on demand. Product rather than the kind of more general merchandise. And so once we realize this, we thought that this could be an interesting thing to add to the tools. So we built an algorithm around this that basically tries to do the exact same sort of thing I just did visually, but algorithmically. We will actually warn you in the tool. So if you type merge seller. In as a topic keyword, you try to have a topic centered on merch seller. We will tell you that actually this topic has multiple intents and. We even tell you what they are merch on Amazon or merchandise stores and we suggest that you pick one of those other keywords rather than targeting the keyword with the ambiguous intent. So taking a step back, that was just a quick introduction to the way that we think about AI Graphite and some examples of what we've been building. And now I will turn it back over to Ethan.