Is Chinese AI generated content any good?

In recent months, a new pig has been herded through the content creation village under the name “AI Content”, sometimes also under the names Jasper, GPT3 or chatGPT. It’s about text content that is written by an Artificial Intelligence and almost looks like it was written by a human.

Such AI systems have the capabilities to do the copywriting of FAQs for a website, write product descriptions, write intro text for a blog post, or write social media posts.

These algorithms work excellently in English and even in other European languages, such as German, Spanish or French.

However, if we look at the user interfaces of the generator tools that offer these services, we also find Chinese in the list of languages offered. Is this the holy grail for European mid-sized companies to save tons of money? From one day to the next, it will no longer be necessary to pay expensive translators or agencies to write technical texts for industrial products? Or is it a big risk?

Some examples of Mandarin Chinese in Simplified Chinese content pieces generated by an AI copywriter

A poem:

Asking Kun Tang for his opinion, he answered this:

So maybe creative tasks are not the strength of chatGPT in Chinese.

Let’s try an e-commerce category introduction text:

Another Chinese colleague, usually responsible for evaluating Chinese texts we receive from clients and producing Chinese and English copy by herself comments:

So, the AI makes some expressional mistakes, a native speaker wouldn’t do.

How about a more B2B like context?

We are asking the AI to produce a short text on explosion-proof floor scales that are intrinsically safe.

Our founder, CEO and native Chinese speaker Kun Tang comments on it:

This is what it looks like at first – not good at all.

But the AI can only be as good as the content it has been trained with (this training process is called “Machine Learning”). The text corpus on which natural language processing ML algorithm was trained was very large for English and much smaller for other European languages. Whether the AI was fed with technical content in Chinese that is relevant for your business is extremely questionable.

Chinese AI Content needs to be proof read and quality checked

So you will never be able to have a somewhat intelligent AI write content and put it online unchecked. You will always need the native language check to ensure quality. In most cases we have tested in, with content that would have been relevant to our current customers, was disastrous. The facts were simply wrong, references were made incorrectly, and the linguistic quality was average at best. At no point would our customers have been satisfied with this content – and neither are we.

What if you do not know it is AI created?

However, there is a big danger: agencies could spring up overnight that claim to be able to write Chinese content natively – at competitive prices below those of the current professional Chinese text agencies / Chinese translation services – by making use of such technological text generation platforms. That can become a big problem:

These agencies will not admit that their texts are written by an AI. They may claim that they work with numerous freelancers from China and that this model explains their competitive prices. Now we don’t want to claim that this wouldn’t be a good business model – but we want to warn that this can be a big lie and the Chinese freelancers may simply be AI texting tools lining their users’ pockets with dollars.

Let’s try to recognize AI created Chinese text automatically

So how do you recognize these AI-written texts? Well, there are a number of tools that have been trained to recognize texts written by another AI. We have tested some of them for you.

First, we had chatGPT explain to us in Chinese what online marketing and SEO in particular is:

Then we copied it to some of the best AI text detection tools available:

https://huggingface.co/openai-detector

https://contentatscale.ai/ai-content-detector/

https://writer.com/ai-content-detector/

None of the tools was able to say, that our text was created by an AI. The closest cam the ai-content-detector from writer.com, which gave our text a 17% probability for being artificially created (that means the chances are 83% that the text was written by a human being).

The reasons for that are most likely, that these AI detection models have mainly been trained on English content. In some random tests, they weren’t able to detect German or Spanish AI created content either, while English chatGPT built content was easily recognized.

So currently, there is no (free) tool based way available for detecting AI created texts. But we at Jademond have created a method to recognized AI created text content. This gives a little head start. An “guessing is it AI” tool only available to our own staff. Of course – we will always still need to evaluate a text “by hand” in order to give good recommendations on what needs to be changed. But this way we can also provide a guess, if the text was eventually automatically created.

Lets conclude

No matter where you get your content from, if it is written in house by a native Chinese speaker, if you have your original English content translated by a certified agency, or if you have a specialized Chinese content agency that produces high quality text for your website or your Social Media profiles, it does always make sense to have it double checked, by another native Chinese speaker.

This is the same as with English content – the four-eyes principle is what makes content great!

Chinese with AI-assistance created content or even fully AI-generated texts at the current state seems to be not good at all, but there is currently no available tool to distinguish artificially created Chinese content from human created Chinese content.

That means you will need a native Chinese speaker in your company to approve all Chinese content created for your business, have a high level of trust in your text / translation agency, that they will only produce high quality content based on your input or option c) to add an additional layer of content quality assurance to your Chinese content production process.

Jademond Digital is a China based company with 90% of native Chinese employees with a high level of experience writing and checking texts for Chinese websites and Social Media channels.

We can fit seamlessly into your existing agency pool. Without artificially denigrating the performance of other agencies, we already work with some clients as a last resort of quality control.

So far, we haven’t had a client who we’ve had to tell that their agency delivers poor quality, yet every now and then we have optimization recommendations that we’re happy to give back, have them incorporated, and incorporate into the texts independently.

Let’s jump on a no-obligation free call and discuss how we can make your (hopefully) good Chinese content even better together.