5 Small Language Model Providers for Local and Specialized AI

Small models are no longer just scaled-down versions of larger LLMs. Some are built for local inference, some for tightly defined production tasks, and others skip general-purpose generation altogether in favor of extraction, classification, or retrieval.

As of September 2026, five SLM providers have become prominent: Fastino, distil labs, Liquid AI, Arcee AI, and Mistral AI. In this comparison, we'll take a look at what each of them actually builds, their models' use cases, and where a different type of small model would make more implementational sense.

Five takes on small language models

Before we get into the specifics of each provider, let's first clarify that there's no universal parameter cutoff for a small language model. distil labs' current student catalog goes from 135M–9B parameters, and sparse models complicate things additionally because their total parameters and active parameters can differ by quite a bit (Arcee's Trinity Mini, for example, has 26B total parameters but activates about 3B per token).

So, in this post, by small-model provider we'll be referring to compact models intended to lower inference requirements or specialize a bounded task. This includes dense SLMs, sparse models with low active parameter counts, distilled students, and compact encoder models such as GLiNER2.5. A 74M extraction encoder and a 14B generative model can both reduce compute relative to much larger general models, even though they're meant for different use cases.

Here's a quick overview of how the five providers we feature here differ by model focus, deployment, and application:

1. Fastino — specialized small models for structured extraction

Fastino's GLiNER family targets structured prediction. GLiNER2.5 can extract entities and relations, classify text, enforce schema constraints, and attach attributes to extracted spans without generating an answer token by token.

Released in August 2026, GLiNER2.5 comes in 74M, 200M, and 300M parameter variants under Apache 2.0. Fastino replaced GLiNER2's span-enumeration mechanism with boundary prediction, which lets the model identify longer spans without a fixed span-width ceiling. The current library also supports long-document chunking and joint entity-relation extraction under a declared schema.

Fastino also publishes dedicated models for PII detection and guardrails, along with larger domain-specific generative models.

Starting with v1.6.0, cognee uses GLiNER in its local memory pipeline. If no usable LLM key has been configured, cognee can switch to the optional gliner_demo extractor and local fastembed embeddings, so text can be processed without an LLM API key. The GLiNER extra is installed separately with cognee[gliner]; it's not bundled into the default Docker image.

In graph-based agent memory, this gives entity and relationship extraction a local path when the extraction schema is constrained enough for a small encoder.

Solid choice for: Entity or relation extraction, classification, PII detection, guardrails, and other schema-defined tasks. GLiNER2.5's 74M–300M models can handle these jobs locally without the compute footprint of a multi-billion-parameter generative model.

Probably not for: Applications that depend on open-ended conversation, long-form generation, or broad reasoning. GLiNER2.5 is built for structured prediction, so those workloads still need a generative model.

2. distil labs — custom small models for stable production tasks

distil labs builds task-specific student models from a production workload. Its training pipeline can start with existing traces or a seed dataset, generate synthetic examples with a larger open-weight teacher, fine-tune a smaller student, and evaluate the student against held-out task data.

The current student catalog includes models from Qwen, Llama, Gemma, SmolLM, and Liquid, starting with SmolLM2 135M and going up to Qwen3.5 9B. The deployed model is the student; the larger teacher is used during training to generate and validate data, then drops out of the production path.

This setup is intended for tasks that have settled into a repeatable pattern. distil labs currently supports classification, question answering, open-book QA, closed-book QA, tool calling, multi-turn tool calling, and chat-oriented tasks. If the existing LLM call has a stable prompt and a measurable output, a smaller student can be trained specifically for that task instead of retaining a frontier model's broader capabilities..

The self-service hosted endpoint is a testing deployment, not a permanent production service. Paid production endpoints are arranged separately. The trained weights can also be downloaded; R&D self-hosting is standard under distil labs' current terms, while commercial self-hosting requires written consent.

Solid choice for: Repeatable production tasks with representative examples and a clear way to measure success. Classification, routing, extraction, QA, and defined tool-calling flows are easier to compress into a student model once the workload has stabilized.

Probably not for: Tasks where prompts, schemas, tools, or expected outputs are still changing frequently. A student learns the task boundary it was trained on, so unstable requirements can mean repeated retraining and less reliable evaluation.

3. Liquid AI — compact models for constrained hardware

Liquid AI develops its LFM family for efficient inference across CPUs, GPUs, NPUs, phones, laptops, and embedded hardware. The current LFM2.5 text lineup includes 230M, 350M, 1.2B, 2.6B, and 8B-A1B models, alongside separate vision-language, audio, encoder, embedding, and retrieval variants.

The 230M and 350M models target highly constrained hardware. At the other end, LFM2.5-8B-A1B uses a sparse Mixture-of-Experts architecture with about 1B active parameters per token, reducing compute per token without reducing the full model to a 1B memory footprint — all 8B parameters still have to be stored.

Liquid also publishes 230M and 350M encoders and a 350M embedding and retrieval model. This lets a local AI stack divide generation, classification, and retrieval among compact models rather than send every operation through one generative model. The same hardware constraints apply to on-device agent memory, where inference and persistent context may both need to stay on local hardware.

Solid choice for: Inference on phones, laptops, vehicles, embedded systems, or other hardware where memory, latency, connectivity, or privacy limit dependence on cloud models.

Probably not for: Cloud-first workloads where general model capability takes priority over local efficiency. Liquid's device-oriented design offers less of an advantage when hardware and connectivity aren't meaningful constraints.

4. Arcee AI — open models with post-training tooling

Arcee AI's Trinity family combines open-weight models with an API and a long-running focus on model post-training. The two smaller Trinity variants are Trinity Nano (download/local-first), with 6B total and 1B active parameters per token, and Trinity Mini (through API), with 26B total and 3B active. Both use sparse Mixture-of-Experts architectures and support 128K context windows.

Nano is aimed at local, edge, and privacy-sensitive deployments, while Mini is intended for cloud, VPC, on-prem, and higher-throughput services. Across the family, Arcee supports multi-turn conversation, tool use, and structured outputs across the model family.

Arcee's model catalog is backed by open-source post-training libraries including MergeKit, DistillKit, and EvolKit. The company publishes foundation models alongside tooling for merging, distillation, and model evolution, so customization extends beyond downloading a checkpoint.

Solid choice for: Applications that need open weights, deployment control, and room to customize a generative model stack. Trinity Nano and Mini retain conversation, tool use, and structured-output capabilities alongside Arcee's post-training tooling.

Probably not for: Tasks that can be handled by a much smaller dedicated model. Even Trinity Nano has billions of parameters, so extraction, classification, or another narrow workload may need substantially less memory and compute with Fastino, Liquid, or a distilled student.

5. Mistral AI — general-purpose compact models with API and self-hosting

Mistral AI's Ministral 3 family comes in 3B, 8B, and 14B parameter sizes. Each size has base, instruct, and reasoning variants, with image understanding and Apache 2.0 weights. The models can be downloaded for local deployment or called through Mistral's hosted API.

All three currently support a 256K context window, along with structured outputs, function calling, document Q&A, chat completions, and batch processing through Mistral's platform. Ministral is the jack of many trades of this list: even their smaller models still handle chat, vision, tools, structured outputs, and reasoning.

Mistral also makes hosted pricing unusually easy to inspect. As of September 2026, their standard rates are $0.10/$0.10 per million input/output tokens for Ministral 3B, $0.15/$0.15 for 8B, and $0.20/$0.20 for 14B. Cached input is priced at one-tenth of the standard input rate.

Solid choice for: A compact generative model that needs to handle conversation, multimodal input, structured output, function calls, and reasoning, with both hosted API access and downloadable weights.

Probably not for: Workloads with tightly defined outputs that don't need broad generative capability. A dedicated encoder or task-specific student can handle extraction, classification, PII detection, and similar jobs with far less compute than a 3B–14B Ministral model.

The workload determines how small you can go

Parameter count is a hollow number if it's not contextualized by the task and deployment. While a known schema might only need a dedicated encoder and a stable production workflow can usually be distilled into a student model, larger tasks still need enough capacity for reasoning, generation, tool use, or multimodal input.

Hardware requirements are also a factor — a 74M extraction encoder can run with a fraction of the resources needed by these generative models because it solves a much narrower problem. Sparse MoE models reduce compute per token, but their full parameter set still affects memory requirements. Benchmark results are easier to interpret once the task and deployment limits are clear.

Optimization here comes down to finding the smallest model that preserves the capabilities the workload actually depends on, then choosing the provider and deployment setup around that constraint.