close

DEV Community

mycute
mycute

Posted on Edited on

The model card is the model: a 3.7 KB PDF that classifies SMS

The model card is the model: a 3.7 KB PDF that classifies SMS

Most model cards describe a model that lives somewhere else. This one carries the weights
in its metadata.

The artifact is a one-page PDF, 3,832 bytes. The visible page is a model card. The
classifier is a 160-word Complement Naive Bayes with 1,579 bytes of int8 weights, stored
as base64url JSON after /Keywords in the PDF's Info dictionary. pdf.html fetches the raw
PDF, reads that string and runs the classifier in the browser. No server, no weights
endpoint. The full SHA-256 is in /TinyAISHA256; the viewer recomputes it and shows a
match or mismatch.

It scores 93.54% on 1,115 held-out SMS messages from the SMS Spam Collection.

Why a PDF

A PDF is a document people already send to each other. It has a visible page for humans and
metadata for tools. Putting the model in the metadata means one file is both the description
and the artifact:

  • the page says what the model is, what it was trained on, how it scored and what it cannot do;
  • the metadata carries the exact weights that produced the score;
  • the viewer page can read the file and classify without a second download.

This is a container, not steganography. Anyone can open the metadata; the point is
provenance, not secrecy. A model card that travels with its model is harder to separate from
its claims.

How it is stored

The Info dictionary contains a normal PDF string:

/Keywords (eyJsIjpbImhhbSIsInNwYW0iXSwidiI6Wy...)
Enter fullscreen mode Exit fullscreen mode

The string is URL-safe base64 of the same JSON used by the QR, favicon and WAV demos. The
page decodes the raw bytes, finds /Keywords (, parses the PDF string (with escapes) and
runs the weights. The core is small:

const text = new TextDecoder("latin1").decode(new Uint8Array(buf));
const m = /\/Keywords\s*\(/.exec(text);
const payload = pdfString(text, m.index + m[0].length - 1);
const raw = b64d(payload);
const model = JSON.parse(new TextDecoder().decode(raw));
Enter fullscreen mode Exit fullscreen mode

The build script starts from demo/model.wav and writes the card text and the metadata from
the same JSON. The card cannot drift from the weights without rebuilding both.

The size accounting

artifact bytes what the bytes are
model JSON 1,579 160 words × 2 labels, int8, scales
QR URL 2,158 page URL + base64 model
favicon PNG 1,519 32×32 RGB pixels, lossless
WAV 1,627 8-bit PCM samples
PDF 3,832 visible card + metadata + page overhead
spreadsheet 10,477 cells and formulas
130 KB data artifact 132,798 500 AG News examples + counts

The PDF pays 2,253 bytes of overhead over the model: one page, a font resource, a content
stream and the PDF structure. The favicon and WAV are cheaper because their formats are
mostly payload. The spreadsheet is larger because a useful sheet needs formulas and visible
cells.

That is the trade: a PDF is the most human-readable of the containers, and it is still under
4 KB.

What the card says

The page is generated from the model, not hand-written:

  • task: SMS spam (ham / spam);
  • data: 5,574 messages, 1,115 held out;
  • accuracy: 93.54% on the held-out split;
  • weights: int8 Complement NB, 160 words, 1,579 bytes;
  • integrity: full SHA-256 in /TinyAISHA256, checked by the viewer;
  • other containers: QR 2,158 B, favicon 1,519 B, WAV 1,627 B;
  • caveats: one 80/20 split, English SMS, bag-of-words, not an LLM.

The caveats are part of the artifact. If you forward the PDF, the limits travel with it.

What this is not

  • Not an LLM. It is a bag-of-words classifier on English SMS.
  • Not a security proof. The metadata is inspectable. A PDF is a lossless container, not obfuscation. The SHA-256 detects a changed model; it is not a signature.
  • Not indestructible. Some PDF editors rewrite or drop metadata when a file is saved. That destroys the model; it does not hide it.
  • Not a production spam filter. One split, 1,115 rows, one machine.

Fragility

A pypdf rewrite kept /Keywords and /TinyAISHA256; rebuilding the PDF without its Info
dictionary — a proxy for print-to-PDF — lost both. That is the PDF trade: the best document
workflow, but not an editable document. The full survival matrix, including PNG, WAV, XLSX,
QR and single HTML, is at
https://byjameson-mini-ai.static.hf.space/survival.html

The same container at 130 KB

The PDF is not only for a 1,579-byte int8 model. The 500-example AG News coreset is
132,798 bytes of JSON in a 178,761-byte PDF; the data is in /TinyAIData and
pdf-130k.html trains the same Complement NB in the browser. It scores 78.3% on the
embedded 400-row test and 74.3% on a fresh sample. That is the same trade at the other end
of the ladder: 46 KB of page/base64/structure overhead, no weights, just examples.

The rest of the ladder

The same 1,579-byte model also lives in a QR URL, a 32×32 favicon, an 8-bit WAV and a
spreadsheet. The point is not that every container is useful; it is that the same behaviour
can be shipped at very different representation costs:

  • a prompt is a few hundred bytes plus a shared base;
  • a 130 KB coreset gets 78.3% on AG News;
  • a 1.7 MB linear model gets 88.5% (91.0% on a fresh sample);
  • a 5 MB adapter changes a shared 4-bit base;
  • a 1.6 KB int8 model gets 93.5% on this narrow SMS task.

The Minimum-Bit Table tracks the measured rows, with the caveats attached:
https://byjameson-mini-ai.static.hf.space/leaderboard.html

Tiny AI Maker now writes a model card for its own browser-trained classifiers too: the
split, a 95% Wilson interval, an optional 5-fold CV with out-of-fold Brier/ECE, per-label
precision/recall/F1, a confusion matrix, the full int8 weight size, the data-artifact size
and the caveats. The 200-row AG News sample is the honest case: 70.0% on the 80/20 holdout,
57.0% across 5 folds (13.0 pp lower), Brier 0.523 and ECE 0.139. A nested temperature fit
(median T 1.5) lowers ECE to 0.074; on SMS it lowers 0.066 to 0.032. The fit is measured,
not applied to the exports. That is exactly why the card shows both. It also exports a PDF card with the weights in /Keywords and a
SHA-256 in /TinyAISHA256; that one is larger (12.8 KB for the 737-word SMS sample) because
it carries the full vocabulary, while this demo uses a 160-word export. The PDF is the same
idea in a document format.

The repo

Everything is in stream-finetune:

If you build a container we did not think of, the table has a submission form.

Top comments (5)

Collapse
 
onelifevibe profile image
OneLifeVibe •

I love the framing that the card cannot drift from the weights because both are generated from the same JSON — that is the part that makes this more than a clever container trick. The size table really sells it too: 93.54% on held-out SMS from a 1,579-byte model, with the PDF costing only 2,190 bytes of overhead for a human-readable page. The honesty about fragility stuck with me as well — a PDF editor rewriting the metadata would destroy the model, so the provenance only holds if the file is treated as an artifact, not edited like a normal document. Have you thought about which of your containers would survive that kind of everyday handling best?

Collapse
 
mycute_97b39c4e34027570f9 profile image
mycute •

Great question — it made me separate "survives copying" from "survives editing".

  • Single HTML and XLSX are the most robust in ordinary handling. The model lives in the content (examples/cells), not in metadata a tool might silently drop. HTML survives copy/email/USB/cloud; a CMS that strips scripts or a minifier can break it. XLSX survives an Excel/LibreOffice resave because the weights are just numbers; Google Sheets import/export is the one to test, since it may rewrite formulas.
  • PDF is the best document workflow but the most fragile metadata. Email/cloud/drive preserve the bytes, but print-to-PDF, "save as", or PDF/A conversion can drop /Keywords. Treat it as an artifact, not an editable document.
  • PNG/WAV survive byte copies but not platform media pipelines: avatars and social images get resized/re-encoded; chat apps transcode audio to Opus/MP3. As attachments they are fine, as "media" they are not.
  • QR URL is excellent in print and fragile in chat: long URLs wrap, link shorteners can drop the fragment, and some scanners cap payload size. The model is in the fragment, so only client-side preservation works.
  • A prompt is tiny but not self-contained; it depends on a shared base.

So: HTML/XLSX for archival, PDF for human-readable provenance, QR for physical sharing. No container survives arbitrary editing; the right question is which transformations are normal in your workflow. I'm turning this into a measured survival table (copy, resave, re-encode, transcode, print) and adding a SHA-256 to the PDF so a rewrite is detectable even when it cannot be prevented.

Collapse
 
mycute_97b39c4e34027570f9 profile image
mycute •

Follow-up after running the experiment — the matrix is live: byjameson-mini-ai.static.hf.space/...

What I measured (same 1,579-byte model, clean digital transformations):

  • XLSX: an openpyxl re-save kept all 483 model cells and 3 formulas. Best archival carrier in this set.
  • Single HTML: byte copy kept the classifier; stripping script tags removed both code and data.
  • PDF: a pypdf rewrite kept /Keywords and /TinyAISHA256; rebuilding without Info (print-to-PDF proxy) lost both. I added a full SHA-256 to the PDF; pdf.html recomputes it and shows match/mismatch.
  • PNG: PNG re-save and metadata strip kept the pixels; JPEG q80 and a 16x16 resize destroyed the model.
  • WAV: PCM rewrite and FLAC kept the samples; MP3 128k and Opus 32k turned the length prefix into garbage.
  • QR: the 2,158-byte URL still decoded after a 10% downscale (161x161), JPEG q5, 4px blur and a 20 degree rotation; dropping the #fragment killed it.

So the practical answer: XLSX/HTML for archival, PDF for provenance (not editing), QR for physical sharing, and PNG/WAV as byte-exact attachments, not media. Caveats: clean digital tests, one machine, no phone camera, no Google Sheets or PDF/A.

Collapse
 
komopdf profile image
KomoPDF •

The survival difference between Info dictionary metadata and file contents comes down to how PDF processors handle document sanitization.

Under ISO 32000-2, the Info dictionary is deprecated in favor of XMP metadata streams under the document catalog. Many modern editors and web viewers rewrite or discard custom Info entries on save, which explains why /Keywords dropped during your print-to-PDF tests.

If you want a model payload to travel inside a PDF while surviving everyday resaves, standard embedded files (/EmbeddedFiles under /Names in the catalog) offer better durability. Storing raw JSON in an embedded file stream also lets you use /Filter /FlateDecode, avoiding the 33% overhead of base64url encoding.

One edge case in pdf.html: searching the raw buffer with /\/Keywords\s*\(/ assumes a literal string in parentheses. PDF syntax allows hex string notation <...> for any string object. If an editor ever touches the file and saves using hex strings or appends an incremental update trailer, matching from the start of the buffer can grab stale or unparsed bytes.

Collapse
 
mycute_97b39c4e34027570f9 profile image
mycute •

Thanks — this is a useful correction, and both points are actionable. The current 3,832-byte PDF uses Info/Keywords because it is simple and a pypdf rewrite preserves it, but the durability argument for /EmbeddedFiles under /Names in the catalog is right; it also lets us use /Filter /FlateDecode instead of paying the 33% base64 tax. I will build a variant with the JSON as an embedded file stream and put the model card and checksum in XMP. On the parser: pdf.html currently regexes the first literal (...) after /Keywords; it does not handle hex strings <...>, and an incremental update could leave a stale first match. I will switch to a catalog/xref-aware parse (or at minimum last-match plus hex-string support) and add tests. The survival matrix already shows a pypdf rewrite preserving /Keywords and a print-to-PDF proxy dropping it; the embedded-file variant is the right next experiment.