Mistral 7B: Open source Model
Mistral-7B is a SOTA language model with a whopping 7.3 billion parameters outperforms llama2–13b on every metric. It represents a significant leap in natural language understanding and generation. The model is released under the Apache 2.0 license, allowing its unrestricted usage.
- Performance Superiority: Mistral-7B surpasses the performance of Llama2–13B on all benchmark tasks and excels on many benchmarks compared to Llama 34B. It also demonstrates competitive performance with CodeLlama-7B on code-related tasks while maintaining proficiency in English language tasks.
- Versatile Abilities: It excels not only in code-related tasks, approaching CodeLlama 7B performance, but also remains highly proficient in various English language tasks.
- Efficient Inference: Mistral-7B utilizes Grouped-query attention (GQA) to enable faster inference, making it suitable for real-time applications. Additionally, Sliding Window Attention (SWA) is employed to handle longer sequences efficiently and economically.
Mistral-7B-Instruct
Fine-Tuning for Chat
Mistral-7B Instruct demonstrates the model’s generalization capabilities through fine-tuning on publicly available instruction datasets. It achieves remarkable performance, outperforming all other 7B models on MT-Bench and competing favorably with 13B chat models.
Evaluation
Mistral-7B vs Llama2
Performance comparisons between Mistral-7B and different Llama models were conducted to provide insights into its capabilities. The benchmarks cover a wide range of tasks, including:
- Comparative Performance: Mistral-7B significantly outperforms Llama2–13B across a wide range of benchmarks, including commonsense reasoning, world knowledge, reading comprehension, and math-related tasks.
- Equivalent Model Size: In reasoning, comprehension, and STEM reasoning (MMLU), Mistral-7B demonstrates performance equivalent to a Llama 2 model more than three times its size. This indicates memory efficiency and improved throughput.
- Knowledge Benchmarks: Mistral-7B excels in most evaluations, it performs on par with Llama2–13B in knowledge benchmarks, possibly due to its limited parameter count.
Mistral-7B consistently outperforms Llama2–13B on all metrics and is competitive with Llama 34B. Notably, it excels in code and reasoning benchmarks.
Mistral 7B performs equivalently to a Llama 2 that would be more than 3x its size. This is as much saved in memory and gained in throughput.
