Bolmo 7B and 1B: The World’s First Fully Open Byte-Level AI Models Explained

The Allen Institute for AI (AI2) has unveiled Bolmo, a groundbreaking family of AI models that introduces a byte-level architecture to large-scale language processing. Unlike conventional large language models (LLMs) such as GPT or Llama, which rely on tokenizers to process text, Bolmo models work directly with raw bytes. This innovation enables more robust handling of noisy, misspelled, or underrepresented languages, positioning Bolmo as a powerful alternative for real-world AI applications.

Bolmo 7B and 1B: The World’s First Fully Open Byte-Level AI Models Explained

What Are Byte-Level Models?

Most LLMs rely on a tokenizer that converts text into tokens based on a pre-defined vocabulary. While effective for standard English, this approach struggles with typos, rare words, or languages outside the tokenizer’s training set.

Bolmo eliminates the tokenizer entirely. By processing UTF-8 bytes directly, the model uses a fixed vocabulary of 256 possible byte values. This allows the AI to handle text at its most fundamental level, preventing unknown tokens and improving performance on noisy or complex datasets. The technique, referred to as “byteification,” retrofits standard transformer architectures to operate natively on bytes rather than tokens.

Bolmo 7B vs Bolmo 1B

Bolmo 7B

• Built on the Olmo 3 7B architecture, the larger Bolmo model delivers robust general-purpose capabilities.

• It performs on par with token-based models of the same size while excelling at character-level tasks such as code parsing, text correction, and handling morphological languages.

• Designed for demanding applications, it provides a competitive open-weight alternative for developers and researchers needing high precision and versatility.

  
  

Bolmo 1B

• Based on the Olmo 2 1B architecture, Bolmo 1B is smaller, faster, and more resource-efficient.

• Ideal for low-compute environments or edge deployments, it offers a practical entry point for byte-level processing while maintaining solid performance.

  
  

Why Bolmo Matters

The release of Bolmo demonstrates that tokenizer-free models can be competitive with traditional LLMs, even at scale. By reading text at the byte level, these models excel in scenarios where tokenization fails—such as processing garbled data, programming code, or languages with complex morphology.

With open-source weights and documentation, Bolmo provides developers and AI researchers with a flexible, high-performance tool to explore innovative applications beyond the limitations of token-based LLMs.

Key Takeaways

• Bolmo is the first fully open, competitive byte-level AI model family.

• It offers robust performance on noisy or rare-language datasets without a tokenizer.

• Bolmo 7B targets high-demand, general-purpose tasks, while Bolmo 1B is optimized for efficiency and lower-resource environments.

• This innovation opens new possibilities for multilingual NLP, code analysis, and character-level AI tasks.

  
  

Bolmo’s launch marks a milestone in AI research, proving that byte-level architectures can deliver real-world value while expanding accessibility for developers worldwide.