Upstage's Solar 10.7B Emerges as World's Top Pre-trained LLM
South Korea’s leading AI company takes global lead on Huggingface Open Leaderboard, with its 10.7-billion-parameter LLM beating Meta, Alibaba, 01.AI, and Mistral AI
Seoul, Dec. 14, 2023– Upstage, a leading South Korean AI startup, proudly announces that its flagship Large Language Model (LLM), Solar 10.7-billion-parameter (10.7B), has secured the top position on the Huggingface Open LLM Leaderboard, surpassing notable Silicon Valley tech giants.
The Open LLM Leaderboard serves as a platform designed to track, rank, and evaluate open-source language models, providing an an objective measure of current state-of-the-art models.
Solar, an acronym for Specialized and Optimized LLM and Applications with Reliability, is an advanced pre-trained generative text model developed by Upstage. Solar 70B, fine-tuned on Llama 2, made an impressive debut last August, exceeding the benchmark score of GPT-3.5 and securing the top position on the Open LLM Leaderboard.
This time, Upstage introduced a new 10.7B English language model, pre-trained on over three trillion tokens. Utilizing a unique depth up-scaling technique based on open-source 7B models, Solar 10.7B surpassed all models on the Open LLM Leaderboard, with an average score of 74.2.
This accomplishment marks a significant milestone for Seoul's AI startup, reinforcing its dominance in the emerging Small Language Model (SLM) category. Despite its compact size, the depth-up-scaled Solar 10.7B has outperformed renowned pre-trained models by major tech giants, surpassing benchmarks set by OpenAI’s GPT-3.5 Turbo (71.07), Meta’s Llama 2 (67.87), and Alibaba’s Qwen 72B (73.6).
In another noteworthy feat, Solar 10.7B surpassed Mixtral 8x7B (72.62) by Mistral AI, a Paris-based AI unicorn that secured $415M in Series A funding. Mistral AI introduced the "mixture of experts" (MoE) model, showcasing performance comparable to Llama 2 and GPT-3.5. This accomplishment further establishes Upstage's prominence in the global AI landscape.
Sung Kim, CEO of Upstage, shared, “"We are delighted to announce our model that surpasses the world’s best AI companies, and we hope that Upstage Solar will be a model for everyone. Upstage is committed to redefining the AI landscape and delivering the most reliable and innovative AI solutions for corporate applications."
Solar 10.7B is now available as an open-source LLM, with both pre-trained and instruction fine-tuned versions under the Apache 2.0 license, enabling unrestricted commercial usage.
Solar excels in private SLM, meeting industry demands for intellectual prowess and practical deployability in various real-world scenarios. Engineered for top performance and security, Solar can be run locally to prevent data leaks and mitigates AI hallucinations through unique dataset training, providing a robust choice for corporate clients.
Korea's leading AI startup Upstage (CEO Kim Seong-hoon) is participating in the global LLM competition in earnest by unveiling its self-developed pre-learning LLM (Large Language Model) 'Solar'. The photo shows Upstage's model taking first place in the Hugging Face Open LLM leaderboard rankings.
