Stability AI has released an exciting new open source large language model called Free Willy 2. This model even outperforms Meta’s recently unveiled LLama 2, demonstrating the rapid innovation possible with open source AI development.
Introducing Free Willy 2 – Open Source LLM to Beat ChatGPT
Free Willy 2 is the latest iteration of Stability AI’s Free Willy line of models. It leverages Meta’s LLama 2 architecture as a base, but improves on the performance through careful fine-tuning using a novel synthetic dataset.
Some key facts on Free Willy 2:
- Uses LLama 2 architecture with 70 billion parameters
- Trained using new high-quality instruction dataset
- Employs “Orca method” for efficient training
- Outperforms LLama 2 on benchmarks
- Released under non-commercial license
How Free Willy 2 Was Created
The Free Willy 2 team utilized a clever training methodology called the Orca method pioneered by Microsoft researchers. This involves teaching a smaller model the step-by-step reasoning of a larger model, rather than just its output style.
The team generated a dataset of 600,000 examples with LLama 2, far smaller than Orca’s original dataset. This greatly reduced computing requirements while still imparting logical capabilities.
Benchmark Results
Across a range of benchmarks evaluating logical reasoning, summarization, and language skills, Free Willy 2 outperformed the vanilla LLama 2 in most tests.
Some key benchmark results:
- 4 point higher average across all tests
- Matches ChatGPT in some logic tasks
- Leads all open source models in combined benchmarks
- Slightly behind LLama 2 in MLMLU benchmark
This demonstrates the potential for open source communities to rapidly advance even huge proprietary models like LLama 2.
Conclusion
With its strong performance and open source accessibility, Free Willy 2 establishes a new standard for large language models. It exemplifies how the open source ecosystem can quickly iterate and enhance models, outpacing even tech giants like Meta.
Free Willy 2 provides impressive language understanding capabilities while minimizing compute requirements. This research pushes towards the dream of capable yet efficient AI that can unlock new beneficial applications.