Elon Musk’s AI startup, X.ai, has unveiled its latest generative AI model, Grok-1.5, which is poised to power the social network X’s Grok chatbot in the near future. According to a blog post by X.ai, Grok-1.5 represents a significant improvement over its predecessor, Grok-1, as evidenced by published benchmark results and specifications.
One notable enhancement in Grok-1.5 is its “improved reasoning,” particularly in coding and math-related tasks. The model achieved notable performance boosts on popular benchmarks, such as the MATH benchmark, where it more than doubled Grok-1’s score, and the HumanEval test, where it scored over 10 percentage points higher in programming language generation and problem-solving abilities.
Despite these benchmark results, it remains challenging to predict how Grok-1.5’s improvements will translate into real-world usage. While benchmarks provide valuable insights, they may not fully capture how average users interact with AI models in practical scenarios.
One observable improvement in Grok-1.5 is its enhanced ability to understand context compared to Grok-1.5. Grok-1.5 can process contexts of up to 128,000 tokens, allowing it to retain information from substantially longer documents and handle longer and more complex prompts while maintaining its instruction-following capability.
Historically, what has distinguished X.ai’s Grok models is their willingness to respond to questions about typically off-limits topics, such as conspiracies and controversial political ideas, and their tendency to adopt a rebellious tone in responses. However, it’s unclear whether Grok-1.5 brings any changes in these areas, as X.ai did not address this in the blog post.
Grok-1.5 will soon be available to early testers on X, accompanied by several new features. Elon Musk has hinted at features such as summarizing threads and replies, as well as suggesting content for posts, which may be introduced with the new model.
This announcement follows X.ai’s decision to open source Grok-1, although without providing the necessary code for fine-tuning or further training. Additionally, Musk recently mentioned that more users on X, particularly those on the $8-per-month Premium plan, would gain access to the Grok chatbot, which was previously exclusive to Premium+ customers paying $16 per month.