Stability AI’s Free Willy 2 Model Sets New Standard for Open Source LLMs

Stability AI has released an exciting new open source large language model called Free Willy 2. This model even outperforms Meta’s recently unveiled LLama 2, demonstrating the rapid innovation possible with open source AI development.

Introducing Free Willy 2 – Open Source LLM to Beat ChatGPT

Free Willy 2 is the latest iteration of Stability AI’s Free Willy line of models. It leverages Meta’s LLama 2 architecture as a base, but improves on the performance through careful fine-tuning using a novel synthetic dataset.

Some key facts on Free Willy 2:

  • Uses LLama 2 architecture with 70 billion parameters
  • Trained using new high-quality instruction dataset
  • Employs “Orca method” for efficient training
  • Outperforms LLama 2 on benchmarks
  • Released under non-commercial license

How Free Willy 2 Was Created

The Free Willy 2 team utilized a clever training methodology called the Orca method pioneered by Microsoft researchers. This involves teaching a smaller model the step-by-step reasoning of a larger model, rather than just its output style.

The team generated a dataset of 600,000 examples with LLama 2, far smaller than Orca’s original dataset. This greatly reduced computing requirements while still imparting logical capabilities.

Benchmark Results

Across a range of benchmarks evaluating logical reasoning, summarization, and language skills, Free Willy 2 outperformed the vanilla LLama 2 in most tests.

Some key benchmark results:

  • 4 point higher average across all tests
  • Matches ChatGPT in some logic tasks
  • Leads all open source models in combined benchmarks
  • Slightly behind LLama 2 in MLMLU benchmark

This demonstrates the potential for open source communities to rapidly advance even huge proprietary models like LLama 2.


With its strong performance and open source accessibility, Free Willy 2 establishes a new standard for large language models. It exemplifies how the open source ecosystem can quickly iterate and enhance models, outpacing even tech giants like Meta.

Free Willy 2 provides impressive language understanding capabilities while minimizing compute requirements. This research pushes towards the dream of capable yet efficient AI that can unlock new beneficial applications.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top