Model cards and system cards, explained

01

What is a model card?

A model card is a short document published with a machine-learning model that says what the model is for, how it was built and evaluated, and where it falls short. Its job is to let people decide whether a model suits their purpose before they rely on it.

The format was proposed by Mitchell et al. in “Model Cards for Model Reporting”, presented at the FAT* conference on fairness, accountability and transparency in January 2019 1. The paper describes model cards as short documents accompanying trained models that provide “benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups”, and that disclose the context in which a model is meant to be used. The authors gave two examples: a model that detects smiling faces in images and one that detects toxic comments in text.

Model cards have a companion for data. “Datasheets for Datasets” proposes that every dataset come with a record of why it was created, how it was collected and what it should and shouldn’t be used for 2. Both rest on the same observation: a model’s behaviour depends on choices its users can’t see, so those choices have to be written down.

On Hugging Face, which hosts many open models, a model card is the README.md file in a model’s repository. Its documentation puts it plainly: “model cards are simple Markdown files with additional metadata” 3.

02

What should a model card contain?

A model card should say what the model is, how well it works and for whom, and what to watch for. Mitchell et al. set out nine sections that between them answer those three questions, as Figure 1 shows.

The central idea is disaggregation. A single accuracy figure averages over everyone a model was tested on, and can hide a group for whom it works badly. Model cards ask for results broken down by the factors relevant to the intended use, such as age group, camera type or lighting, and by combinations of them 1. The paper accepts that training data may not be shareable, and asks at least for how it is distributed across the same factors.

Cards on Hugging Face add a machine-readable header. A YAML block at the top of README.md records fields such as the licence, languages, datasets and base model, and can carry evaluation results in a structured model-index that the Hub displays on the model page 3. For a fine-tuned, adapted, merged or quantised derivative, base_model names the original, so a derived model can be traced to the card that describes its parent.

---
license: apache-2.0
base_model: org/base-model       # the model this one derives from
base_model_relation: quantized   # or finetune, adapter, merge
datasets:
  - org/training-data
model-index:                     # structured evaluation results
  - name: org/this-model
    results: [...]
---

Naming the sections is the easy part; filling them in is not. In 2024, Liang et al. analysed 32,111 model cards on Hugging Face. Most models with substantial downloads had a card, but the sections on environmental impact, limitations and evaluation were the least often filled out, while the training section was the most consistently completed 4. The parts a reader most needs in order to judge a model are the parts most often missing.

03

What is a system card?

A system card documents a deployed AI system rather than a single model: the models, plus the instructions, tools, retrieval, safeguards and data flows around them, and how the whole was tested. It describes the product people actually use.

The difference matters because the same model behaves differently in different systems. A system prompt changes what it says. Retrieval changes what it knows. A context limit changes how much of a conversation it sees, and safeguards stop some outputs before anyone reads them. None of this appears on a model card. OpenAI’s system card for GPT-4, published in March 2023, is one example: it describes safety challenges found in testing and a deployment process spanning “measurements, model-level changes, product- and system-level interventions (such as monitoring and policies), and external expert engagement” 5.

A useful system card answers a fixed set of questions: which models and versions run, and with what settings; what instructions they are given; which tools and sources they can use; what is stored or sent, where and for how long; which safeguards sit around the models; what the system is known to get wrong; and how it was tested. Each answer should be specific enough for a reader to check. Table 1 compares the two kinds of document.

04

How do you read a model card critically?

Read a model card as evidence, not as a specification. Check what was measured, on what data and for which version of the model, and notice what the card leaves out. Five questions cover most of it.

1. Is this the model you will run? Published results usually describe the developer’s own release. A quantised build, a shorter context window, a different runtime or a different system prompt makes a different system, and the card’s numbers don’t carry over automatically.
2. Are results broken down? Look for results by group and condition, with sample sizes and some measure of uncertainty, rather than a single average.
3. Was the test data kept separate? If benchmark questions leaked into the training data, a high score may reflect memory rather than skill. A careful card says how it guarded against this.
4. Are limitations and out-of-scope uses stated? They are among the sections most often left empty 4, and they are the ones that tell you when not to use the model.
5. Who wrote it, when, and on what terms? A card is a dated snapshot, usually written by the model’s developer. Check the version, the date and the licence, which governs what you may do with the model, and read independent evaluations alongside the card where they exist; ‘AI safety evaluations, explained’ describes what those involve.

05

What does Google’s Gemma 4 model card contain?

Google DeepMind’s Gemma 4 model card is a current example for an open-weight model family, and two Brello models are based on models it describes. It was last updated on 30 July 2026, and we checked it on 5 October 2026 6.

Its structure extends the original nine sections for a modern model. It opens with a models overview and a specification table for each size, then gives benchmark results, core capabilities and best practices for prompting and sampling. Model data covers the training dataset and its preprocessing. Ethics and safety covers the evaluation approach and its results. Usage and limitations covers intended usage, limitations, ethical considerations and risks, and benefits. The card names Google DeepMind as its author and Apache 2.0 as the licence.

It covers several sizes, from E2B and E4B, two small language models it describes as targeting mobile and edge devices, to larger models for consumer GPUs and workstations. For E4B it lists 4.5 billion effective parameters, 8 billion with embeddings; for E2B, 2.3 billion, or 5.1 billion with embeddings; both have a context length of 128K tokens. Three sentences show the habits of a careful card. On benchmarks: “Evaluation results marked in the table are for instruction-tuned models.” On safety testing: “All testing was conducted without safety filters to evaluate the model capabilities and behaviors.” And on factual accuracy, models “are not knowledge bases. They may generate incorrect or outdated factual statements.”

Read with the questions above, the card also shows what a model card can’t tell you. Brello Pro and Brello Vision run LiteRT-LM builds of Gemma 4 E4B and E2B, downloaded from Hugging Face, with a 4,096-token context window, answers of up to 1,200 tokens (2,048 with Think harder) and Brello’s own instructions and safeguards. The card’s benchmark scores and its 128K-token context describe Google’s instruction-tuned models, not those builds as Brello runs them, so Brello doesn’t present them as its own.

06

What does the Brello 1.0 system card cover?

The Brello 1.0 system card documents the app as a whole rather than a single model, because what people use is a system: three on-device models, their limits, the instructions and safeguards around them, and every route by which data leaves the phone.

Each model also has its own page in the model-card pattern. Brello Pro’s page, for example, gives its base model, licence, download size, recommended phone memory and runtime, and links to Google’s model card for Gemma 4. Brello Super Intelligence is in development and has no published specifications, so it has no model card or system card. The Brello Charter commits us to evaluate each new capability before release and to publish an evaluation summary with it (commitment 04).