Smarter AI Thanks to Specialized Human Trainers: A Behind-the-Scenes Look
In the past, training AI models like ChatGPT and its competitors involved vast teams of low-cost workers helping machines learn simple distinctions, such as identifying whether an image was of a car or a carrot. However, as AI models have become more advanced, the need for highly specialized human trainers has grown exponentially. Experts in various fields, from history to medicine, are now integral to shaping the next generation of AI.
Cohere co-founder Ivan Zhang explained that a year ago, undergraduates were enough to teach AI general improvements. But today, licensed professionals such as physicians and financial analysts are needed to ensure the models perform effectively in specialized environments. Cohere, a key rival of OpenAI, works with Invisible Tech, a startup that supplies thousands of trainers to help AI models reduce errors, known in the industry as “hallucinations.”
Invisible Tech, founded in 2015, initially focused on workflow automation but pivoted to AI training after OpenAI approached them in 2022. This partnership came about because OpenAI’s early ChatGPT models were prone to generating incorrect information, and they needed advanced human feedback to address this issue.
Invisible’s founder, Francis Pedraza, highlighted the company’s role in providing specialized human trainers to most of the big players in the generative AI (GenAI) space, including OpenAI, Cohere, and AI21. Invisible’s network now consists of 5,000 experts worldwide, many of whom hold advanced degrees. Depending on the complexity of the task, Invisible pays its trainers up to $40 per hour, with other companies in the space offering even higher rates for niche expertise.
The issue of AI “hallucinations” has become a significant challenge for the industry. These occur when AI models generate factually inaccurate information, undermining their reliability, especially for businesses. To address this, companies like OpenAI run constant experiments to improve AI performance, relying heavily on human trainers with deep domain expertise to refine the models.
Invisible’s role extends beyond just providing human labor; they also help manage the increasingly large and complex tasks required to train AI. As Pedraza put it, OpenAI has some of the world’s best computer scientists, but they may not be experts in fields like Swedish history or biology. That’s where specialized trainers come in, filling knowledge gaps and making the models more versatile.
Other companies in this space, such as Scale AI, which was last valued at $14 billion, also supply training data and have ventured into providing AI trainers. However, Invisible remains a key player, having been profitable since 2021 despite only raising $8 million in primary capital. Pedraza noted that Invisible’s unique ownership structure allows them to maintain 70% control within the team, with a secondary market valuation of around half a billion dollars.
The rise of specialized trainers marks a significant shift from earlier days of AI training, which mostly involved low-paid, less-qualified workers from African and Asian countries. Today, demand for trainers with deep knowledge across a range of subjects and languages is creating a well-paid niche, opening doors for experts in various fields to contribute to AI development without needing coding skills.
As AI companies continue to push the boundaries of their technology, the market for specialized trainers is growing. Zhang, from Cohere, mentioned his inbox is flooded with inquiries from new firms offering similar services, underscoring the increasing importance of human expertise in the evolution of AI.
