falcon-7b
- User
falcon-7b
- Assistant
"Falcon-7B" refers to a series of open-weight large language models developed by the Technology Innovation Institute (TII) in Abu Dhabi. These models are part of the Falcon series, which includes models with 7 billion (7B), 40 billion (40B), and 180 billion (180B) parameters. The Falcon-7B model is a causal decoder-only model trained on a diverse, high-quality corpus predominantly assembled from web data. It was introduced in November 2023 and has been trained on over 3.5 trillion tokens of text, making it one of the largest openly documented pretraining runs at that time. (arxiv.org)
In October 2024, TII introduced Falcon Mamba 7B, a new base large language model based on the novel Mamba architecture. This model was trained on 5.8 trillion tokens with carefully selected data mixtures. As a pure Mamba-based model, Falcon Mamba 7B surpasses leading open-weight models based on Transformers, such as Mistral 7B, Llama3.1 8B, and Falcon2 11B. It is on par with Gemma 7B and outperforms models with different architecture designs, such as RecurrentGemma 9B and RWKV-v6 Finch 7B/14B. Due to its architecture, Falcon Mamba 7B is significantly faster at inference and requires substantially less memory for long sequence generation. The weights of Falcon Mamba 7B are publicly available on Hugging Face under a permissive license. (arxiv.org)
These models are designed to be competitive with other leading language models and are intended to foster open science and accelerate the development of an open ecosystem of large language models.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.