WDF*IDF for Baidu: Complete Is Not the Same as Better

Using every word a WDF*IDF tool recommends will make your page complete. It will also make it average, because every competitor who runs the same tool ends up with the same list. The skill is in the second step: finding what you can add that no other page on the topic has.

WDF*IDF is a method for finding out which words a page on a given topic is expected to contain. Western SEO teams have used it since around 2012. For Baidu it is harder and more interesting, because Chinese text has no spaces to tell software where one word ends and the next begins. We recently built an internal tool so our SEO specialists can apply the method to Baidu, and this article explains the thinking behind it.

WDF*IDF scores a word by asking two questions: how prominent is it on this page, and how rare is it across everything else?

Think of a recipe. Salt appears in almost every dish, so finding salt in a recipe tells you nothing about what it is. Saffron appears in very few, so finding saffron tells you a lot: probably paella or risotto. WDF*IDF is a way of giving saffron a high score and salt a score of nearly zero.

The two halves have names. WDF (within document frequency) measures how often a word appears on the page, relative to all the other words, with a logarithm applied so that one heavily repeated word cannot dominate (IONOS explains the German-SEO variant). IDF (inverse document frequency) measures rarity: the fewer documents contain a word, the higher its score. This idea goes back to Karen Spärck Jones, who described it in 1972 as term specificity (Wikipedia). Multiply the two and you get the weight.

The numbers are illustrative, and the rarity score is the logarithm (base 10) of total pages divided by pages containing the word. "The" is on nearly every page, so it scores almost nothing. "Kettle" is the topic, so it scores highest. "Descaling" makes up less than half a percent of the words, yet it earns more than half of the weight of "kettle", because so few pages mention it. Real tools use a logarithmic WDF and far larger samples, but the logic is the same.

In SEO, WDF*IDF becomes a competitor comparison: the tool takes the pages that already rank for your keyword and shows which terms they use that your page does not.

The scientific version compares a page with a huge collection of documents. SEO tools shrink that collection to the top search results for one query, so the output is an approximation. It is still useful, and the report reads simply: "the ranking pages mention this term often, and yours never does."

The term WDF was used in information science in the early 1990s (again per IONOS). SEO practitioners have promoted the formula since around 2012 as a smarter replacement for keyword density, which only counts how often one keyword appears (Holistic SEO, Textbroker). Today's content-optimization platforms follow the same pattern: they scrape the top-ranking pages and grade your draft against them (Clearscope, Surfer vs Clearscope). Whatever the brand name, the input is the same: what is already ranking.

Google has downplayed TF-IDF as a ranking tool, and term-gap analysis still improves pages, because it measures something search engines do care about: topical completeness.

Google's John Mueller has called TF-IDF a fairly old metric and said Google uses many information-retrieval techniques, not just this one. He pointed out that outsiders cannot reproduce it, because it depends on statistics from the entire web, and advised focusing on the website and its users instead (Search Engine Journal). We know of no comparable public statement from Baidu either way.

Three things explain why the method helps anyway:

The evidence for the size of the effect is modest. Comparison articles quote correlations between tool scores and rankings of about 26% for Surfer and 17.5% for Clearscope, and note that neither guarantees rankings (Genesys Growth). We have not verified how those figures were produced, but the pattern fits: term coverage helps, and it does not decide.

Baidu is not a word counter either. As we noted in our Baidu ranking factors study, Baidu's investment in its ERNIE language model suggests that it analyses content with far more than word frequencies.

Before a Chinese page can be scored, software has to decide where the words are, and different decisions produce different scores.

In English, spaces do most of the work. German glues words together ("Kaffeemaschine" is one word for coffee machine), so tools need to split them. Chinese goes further: there are no spaces at all, and no obvious marks between characters showing where a word ends (PMC). Researchers name three core difficulties: vague word boundaries with no authoritative standard, ambiguous words, and unknown words. This is one reason why tools built for spaced languages cannot simply be pointed at Chinese pages.

Specialist vocabulary is even less forgiving. In research on quality-management texts, a segmenter built for everyday language split the product term 大直径钢制管法兰 into three parts: 大直径 (large diameter), 钢制 (steel) and 管法兰 (pipe flanges). The split fails to express what an engineer means, which is one product (PMC). In the same study's test on Chinese flange standards, the popular segmenter jieba got only 5 of 184 long technical terms (five characters or more) right (PMC). Segmenters like jieba are dictionary-based: they scan a dictionary for known words and use a statistical model to guess unknown ones (arXiv). So the dictionary decides which words exist.

For WDF*IDF this has three practical consequences:

In our Baidu SEO Ranking Factors Correlation Study, the pages that ranked were text-rich, image-heavy and light on repeated keywords, but the averages hide a lot. We analysed the results for 10,000 Chinese keywords, looking at the top 20 positions of each, with data collected in 2023 (summary on Search Engine Journal, full study).

These are useful for sanity-checking a draft. They are poor instructions for a specific page, and the study itself shows why:

The lesson for term analysis: run it per query, on the results that actually rank for that query, and read the output as a description of that one search result page. Averages tell you the neighbourhood, and your topic decides the street.

A page that uses every recommended term is complete, and it is also a close relative of the pages that already rank.

Every team running a tool draws from the same source, the current top results. Follow the report and you converge on the middle of the results page: similar headings, similar terms, similar depth. Tool vendors describe it this way themselves. One comparison explains that if the top results for a query average a certain length, heading count and set of semantic terms, the tool steers your content towards those metrics (Stackmatix).

One SEO agency frames the same idea geometrically: coverage tools pull a page towards the cluster of existing results, while differentiation moves it away from that cluster, and a winning page needs both (Searchbloom).

Completeness is the entry ticket, not the prize. To a searcher, and to an AI-generated answer, an eleventh page that repeats the first ten adds nothing. Being average takes away the reasons to choose you: nothing new to cite, nothing to click for, nothing to link to.

The formula also cannot see quality. It counts words. It does not know whether a claim is true, whether the author has done the work, or whether the page loads. Practitioners point out that even a perfectly formula-driven text will not make up for weak content, bad backlinks or a page that is not mobile-friendly (Ben Moll).

The extreme case shows where following the numbers ends. A 2017 Chinese SEO article proposed taking an article on the benefits of ginseng, replacing the herb's name with Chinese yam, and raising the frequency of the new term, so that the text would count as unique for that keyword (Alibaba Cloud Developer Community). That is the formula used as a hack: statistically tidy, with nothing new to say. We mention it as a warning, not a method. Agency guides report that Baidu can detect plagiarism and content spinning (Elite Asia).

Treat the term report as the floor of the page, and spend the effort it saves on what only you can add.

Western SEO often calls this idea information gain, after a Google patent about scoring how much new information a page adds. We treat it as a framework, not a proven signal. One analysis reads the patent as a session-level mechanism that scores new information against what one user has already seen, not as a global ranking signal, and notes that it may not be a confirmed ranking signal in Google's documentation (Searchbloom). We know of nothing showing that Baidu uses it. The logic holds without the patent: a page that only repeats what is already there gives a reader no reason to stay.

Term analysis is a useful approximation, and it helps to be clear about how solid each piece of the argument is.

What we can say about adoption. Chinese-language SEO writing has discussed TF-IDF together with word segmentation for years (Alibaba Cloud Developer Community, 2017), and large Chinese tool platforms such as 5118 offer keyword and word-frequency features, including high-frequency word extraction and competitor analysis. What we could not find is a public tool that compares a page against the terms of the pages currently ranking on Baidu, the way Western content tools do for Google. That is the gap our internal tool fills for our team.

Our SEO specialists now use an internal tool to run this analysis for Baidu, query by query. The tool gives us the floor of the page. The part that decides whether the page is worth ranking is still human: what we know, have measured or have seen that the other ten pages have not.

[Editor's note, delete before publishing: add two sentences on what the tool shows, a screenshot with anonymised data, and a link to the contact page.]

Related reading: our step-by-step guide to Baidu SEO, the Baidu ranking factors study, and our guide to Chinese GEO on how AI assistants choose which pages to cite. For hands-on help, see our Baidu SEO services.