This article was originally published on ITProPortal Digital platforms and the post-language economy by Denis Gachot.
The world can get even smaller, and new technology is making that happen.
When we imagine international companies, we think large, publicly traded conglomerates that have substantial resources and funds to facilitate operations on opposite ends of the globe. But that is changing.
Already the Internet has shrunk the world so that small companies now rely on software engineers in Pakistan and marketing agencies contract with graphic designers in the Philippines. But the world can get even smaller, and new technology is making that happen.
Today, the new frontier is language. Individuals have a plethora of platforms that allow them to access consumers all over the globe and work with other companies in faraway places – if only they could speak the same language. In an ironic twist, language has turned from something that first facilitated human cooperation and growth, to something that currently impedes our ability to work together.
Technology may finally be ready to abolish that barrier forever. It is somewhat remarkable that in 2017, more than 20 years after widespread use of the Internet began, we still rely almost exclusively on humans to translate language in commercial formats. But translation bears all of the earmarks of those functions that artificial intelligence ought to be capable of replicating, and a technology called Neural Machine Translation (NMT) does just that.
Contextual translation ability
By leveraging its contextual translation ability alongside its deep learning functions, NMT has achieved historic results in the journey to a post-language economy. In a side-by-side comparison with human translators, in a technical domain translation for English-Korean, SYSTRAN’s NMT translations were preferred 41 per cent of the time. That success is achieved by advancing language translation beyond rule-based translation methods.
Before NMT, machine translation models – known as rules-based or ‘phrase-based’ – were only able to reference five to seven words at a time when determining meaning. Each language pair has its own linguistic challenges, but it made the translation for certain languages, like Japanese, more challenging because you need to know the entire sentence to put all of the words into context.
Let’s say a colleague forwards you an urgent email, and it includes this sentence: パリに出張の時に私はCEOに会いました.
With a phrase-based machine translation, you would receive this output: ‘I met Paris in the CEO trip doing business.’ With NMT, you would get: ‘I met the CEO when I was in Paris on a business trip.’
In Japanese, main verbs are placed at the end, so you need to reference the end of a sentence to make sense of the phrases within it. NMT processes the entire sentence (and soon paragraph) from end-to-end without intermediate stages.
That is why context is so important. The effect of machine translation being able to better understand context results in a huge jump in BLEU score (the industry measurement for accuracy). SYSTRAN’s Pure Neural Machine Translation (PNMT) program has seen increases in all 61 language pairs and where we see the biggest increase is in Asian languages. For some languages, we saw a jump of 200 per cent in BLEU score.
With machine translation, we have a metric called ‘Gisting,’ as in ‘you get the gist.’ In addition to this metric, we test whether or not a user can solve their problem with the translated output. Were they able to search a FAQ and customise a piece of software with the answer? Were they able to search a digital database of products and images and find what they were looking to purchase? If yes, then they got the gist.
“Gisting requires extensive post-editing. NMT has moved us into fluency. What fluency allows is the ability to read and understand so that you no longer need to post edit,” says Ken Behan, V.P. of Sales and Marketing. NMT is allowing us to focus on ‘meaning’ and ‘fluency’ scores. With fluency, we ask if the translation sounds like a native speaker wrote it. With fluency and meaning, we can ask:
Were they able to understand a review and make an educated decision? Were they able to read the manual specs and assemble a piece of heavy machinery? Were they able to find a product, read the description and purchase what they were expecting?
Refer to the English translations above. With the first sentence, did you get the gist? Yes! You could infer someone went to Paris on something business related.
With the second, did you understand the meaning and the fluency? Yes, you can understand what kind of trip it was and what happened.
Neural Machine Translation will further advance traditional MT solutions and create new ones in communication, customer support, e-Learning, eDiscovery, compliance and user-generated content to name a few. Also, early-adopting linguists using NMT are already increasing their productivity.
Similar to the human brain, the neural machine translator learns through a process in which the machine receives a series of stimuli over several weeks. This development, based on complex algorithms at the forefront of Deep Learning, enables the PNMT engine to learn, generalise the rules of a language from a given translated text, and produce a translation close to human levels of competency.
You can think of NMT as part of your international go-to-market strategy. In theory, the Internet erased geographical barriers and allowed players of all sizes from all places to compete in what we often call a ‘global economy,’ But we’re not all global competitors because not all of us can communicate in the 26 languages that have 50 million or more speakers. NMT removes language barriers, enabling new and existing players to be global communicators, and thus real global competitors. We’re living in the post-internet economy, and we’re stepping into the post-language economy.
The difference between previous language technologies and what NMT can do today is remarkable, and business leaders should take note. Just ask yourself who you would want to do business with: the guy that says, “I met Paris in the CEO trip doing business,” or the one that says, “I met the CEO when I was in Paris on a business trip.”
SYSTRAN’s team is setting private meetings for an exclusive view of the PNMT concept. For more information, contact Craig Stern at [email protected].