DEV Community: Libme

Postgres Full-Text Search vs Meilisearch vs Typesense: Do You Need a Search Service for a Small App?

Libme — Mon, 09 Nov 2026 21:59:04 +0000

If your app already runs on Postgres and you need a search box over a few hundred thousand rows, start with Postgres full-text search plus pg_trgm. Add Meilisearch or Typesense when you need typo tolerance as-you-type, faceted filtering with instant counts, or relevance tuning that non-engineers can adjust. Until one of those is a real requirement, a second datastore is mostly a second thing to keep in sync.

What does Postgres full-text search actually give you?

More than people assume. Stemming, stop words, weighted fields, phrase search, ranking, and a websearch_to_tsquery parser that understands quoted phrases and -exclusions the way users type them. As of late 2026 this is all core Postgres, no extension needed.

A setup that works well for a typical articles table:

ALTER TABLE articles ADD COLUMN search tsvector
GENERATED ALWAYS AS (
  setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
  setweight(to_tsvector('english', coalesce(body,  '')), 'B')
) STORED;

CREATE INDEX articles_search_idx ON articles USING GIN (search);

SELECT id, title, ts_rank(search, q) AS rank
FROM articles, websearch_to_tsquery('english', 'postgres "connection pool" -mysql') AS q
WHERE search @@ q
ORDER BY rank DESC
LIMIT 20;

The generated column keeps the index in sync with the row inside the same transaction. That is the real advantage over an external service: there is no indexing pipeline that can fall behind or silently drop a document.

Takeaway: Postgres full-text search gives you transactionally consistent, weighted, stemmed search with zero new infrastructure.

Where does Postgres search fall short?

Three places, and they are the ones users notice.

Typos. Searching postgre or kuberentes returns nothing, because full-text search matches lexemes, not approximate strings. The standard patch is pg_trgm:

CREATE EXTENSION IF NOT EXISTS pg_trgm;
CREATE INDEX articles_title_trgm ON articles USING GIN (title gin_trgm_ops);

SELECT id, title, similarity(title, 'kuberentes') AS sim
FROM articles
WHERE title % 'kuberentes'
ORDER BY sim DESC
LIMIT 10;

This works for short fields like titles and names. It is a poor fit for fuzzy matching inside long bodies, and tuning pg_trgm.similarity_threshold is something you do by trial and error.

Ranking quality. ts_rank does not account for how rare a term is across the corpus in the way BM25-style engines do. For small, homogeneous data you may never care. For mixed content, results can feel arbitrary, and the fix is hand-written boosts in SQL.

Faceting. Counting results per category for every query means a GROUP BY over the whole match set. That is fine at tens of thousands of matches and gets slow beyond that.

Takeaway: Postgres fails at the user-facing polish (typos, ranking, facet counts), not at the correctness of matching.

How do Meilisearch and Typesense differ?

Both are single-binary, open-source search engines aimed at the "instant search box" use case, both ship typo tolerance on by default, and both have hosted cloud offerings if you do not want to run them. The differences that matter in practice:

	Postgres FTS + pg_trgm	Meilisearch	Typesense
New infrastructure	None	One service	One service (cluster for HA)
Typo tolerance	Only via `pg_trgm`, manual	Built in	Built in
Index lives in	Your database	Disk (memory-mapped)	RAM
Consistency with your DB	Transactional	Eventually, you sync it	Eventually, you sync it
Facets / filters	Hand-written SQL	Built in	Built in
Ranking control	SQL expressions	Ordered ranking rules	Per-field weights, sort expressions
Main drawback	Polish, tuning effort	Another sync pipeline	RAM sizing, another sync pipeline

Typesense holding its index in RAM is why it is fast, and it is also the cost driver: your dataset size becomes your memory bill. Meilisearch is lighter on RAM for the same corpus in my experience, but check the current license terms for any specific feature you depend on, since parts of the ecosystem have moved between licenses over time.

If you want the managed, zero-ops version of instant search with typo tolerance, Meilisearch Cloud is the option that removes the hosting question while keeping the engine simple. If you want predictable low-latency lookups and your dataset comfortably fits in memory, Typesense is the one built around that assumption.

Takeaway: Meilisearch and Typesense buy you polish, and the price is a sync pipeline plus one more thing to operate.

What is the hidden cost of adding a search service?

Keeping it in sync. Every create, update, and delete in Postgres must reach the engine, and the failure mode is quiet: a deleted record that still shows up in results, or a new one that never appears.

The minimum viable pattern is an outbox table written in the same transaction as the change, drained by a worker:

CREATE TABLE search_outbox (
  id bigserial PRIMARY KEY,
  article_id bigint NOT NULL,
  op text NOT NULL CHECK (op IN ('upsert', 'delete')),
  created_at timestamptz NOT NULL DEFAULT now()
);

The worker reads in batches, pushes to the engine, then deletes the processed rows. You also need a periodic full reindex, because you will eventually find drift. That is roughly a day of work plus ongoing monitoring, which is the honest comparison against the afternoon Postgres FTS takes.

Takeaway: The search engine is cheap; the sync pipeline and its drift detection are what you actually pay for.

Which one should you pick?

Your situation	Pick
Under roughly a few hundred thousand rows, users search by exact-ish terms	Postgres FTS
Short fields (names, titles) where typos matter	Postgres FTS + `pg_trgm`
Search-as-you-type with typo tolerance is a product requirement	Meilisearch or Typesense
Many facets with live counts	Meilisearch or Typesense
Dataset is small enough to fit in RAM and latency is critical	Typesense
You have no one to operate another service	Postgres, or a managed cloud plan

These size boundaries are rules of thumb, not benchmarks. Measure your own query latency with EXPLAIN (ANALYZE, BUFFERS) before deciding the database is the bottleneck.

FAQ

Is Postgres full-text search good enough for production?
Yes, for keyword search over moderate datasets it is production-grade, with GIN indexes and weighted ranking. It becomes a poor fit when you need typo tolerance in long text or live facet counts.

How do I add typo tolerance to Postgres search?
Enable the pg_trgm extension, add a GIN trigram index on the field, and query with the % similarity operator. It works best on short fields like titles.

Meilisearch or Typesense: which is easier to start with?
Both run as a single binary and take minutes to try locally. Meilisearch is usually lighter on memory, while Typesense keeps its index in RAM, so size your server to your data.

Bottom line

If you already have Postgres and a search box, use tsvector with a GIN index and add pg_trgm for names and titles; you can ship that in an afternoon and it stays consistent with your data. Move to Meilisearch or Typesense when typo-tolerant instant search or faceting becomes something users complain about, and budget for the outbox and reindex job from day one. Pick the engine after you have measured what your data size and latency actually need.

GitHub Actions "Resource not accessible by integration": Fix It With the Smallest GITHUB_TOKEN Permission

Libme — Sun, 08 Nov 2026 21:54:39 +0000

The error means the GITHUB_TOKEN your job runs with lacks the permission for the API call it just made. Fix it by adding a permissions: block to the specific job with only the scope that call needs (for example pull-requests: write to comment on a PR). Don't flip the whole repository to "read and write" just to make the red X go away.

What does the error actually look like?

Depending on the action or CLI, you will see one of these in the job log:

HttpError: Resource not accessible by integration
RequestError [HttpError]: Resource not accessible by integration - https://docs.github.com/rest/...
remote: Permission to org/repo.git denied to github-actions[bot].
fatal: unable to access 'https://github.com/org/repo/': The requested URL returned error: 403
gh: Resource not accessible by integration (HTTP 403)

All of them come down to the same thing: GitHub authenticated the request as the workflow's app token, and that token's permission set did not include what the endpoint requires. It is an authorization problem, not a bad token, so regenerating secrets will not help.

Takeaway: a 403 from github-actions[bot] is a missing permission scope, never an expired credential.

Why did it work last month and fail now?

Three things commonly change underneath you.

Default token permissions. Each repository (and organization) has a setting under Settings → Actions → General → Workflow permissions. Newer repositories default to read-only for the token; older ones may still have the permissive default. A repo created or migrated after the default changed behaves differently from your old one.
An explicit permissions: key. The moment a workflow or job declares any permission, every scope you did not list is set to none. Adding id-token: write for a cloud login quietly removes contents: read, and then actions/checkout fails on a private repo.
Who triggered the run. Workflows triggered by a pull request from a fork get a read-only token, and by default no secrets. Runs triggered by Dependabot also get a read-only token by default. No permissions: block can widen that for a pull_request event from a fork.

Takeaway: the same YAML can get different tokens depending on repo defaults, the trigger, and the PR author.

Which permission does each step need?

This is the part the error message never tells you. As of late 2026, these are the scopes I reach for most:

What the step does	Permission needed
`actions/checkout` on a private repo	`contents: read`
Push a commit, create a tag or release	`contents: write`
Comment on or label a pull request	`pull-requests: write`
Create or comment on an issue	`issues: write`
Push an image to GitHub Container Registry	`packages: write`
Request an OIDC token for cloud login	`id-token: write`
Upload SARIF to code scanning	`security-events: write`
Deploy to GitHub Pages	`pages: write` and `id-token: write`
Write commit statuses or check runs	`statuses: write` / `checks: write`

One trap: PR comments go through the issues API under the hood, so on some endpoints issues: write also works, but pull-requests: write is the one that matches intent and is what most actions document.

How do I scope it to one job?

Set the default low at the top of the workflow, then raise it only in the job that needs it:

name: pr-size-comment
on: pull_request

permissions:
  contents: read

jobs:
  comment:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - name: Post a comment
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          gh pr comment "${{ github.event.pull_request.number }}" \
            --body "Checked by CI."

The workflow-level contents: read means any job you add later starts with the minimum. The job-level block replaces (not merges with) the workflow-level one, which is why contents: read is repeated there.

Takeaway: declare read-only at the top, grant write per job, and repeat every scope the job still needs.

Why does it still fail on fork pull requests?

Because the restriction is deliberate. A fork's code is untrusted; if it could run with a write token or your secrets, anyone could open a PR that exfiltrates them. So a pull_request run from a fork cannot comment, label, or push, whatever your YAML says. In my testing, the confusing part is that the same workflow succeeds on branches inside the repo and fails only for outside contributors.

Your options, from safest to riskiest:

Split the work. Let the untrusted pull_request run build and test, save the results as an artifact, and have a second workflow triggered by workflow_run (which runs in the base repo's context) download the artifact and post the comment. This keeps untrusted code away from the write token.
Skip the comment for forks. Guard the step with if: github.event.pull_request.head.repo.full_name == github.repository, so internal PRs get the comment and forks just pass quietly.
pull_request_target. It runs with a write token against the base branch, but if you check out and execute the PR's code in that job you have handed the keys to the contributor. Use it only for steps that never run PR-supplied code.

The same logic applies to Dependabot PRs: they get a read-only token and a separate Dependabot secrets store. If an auto-merge or comment workflow fails only on Dependabot branches, that is why.

Takeaway: for fork PRs, do not fight the read-only token; move the privileged step into a trusted workflow_run job.

What about reusable workflows and organization policy?

Two more causes that look identical in the log:

Reusable workflows can only reduce permissions. A called workflow inherits the caller's token and can downgrade it, but it cannot grant itself something the caller did not allow. Fix the permissions: on the calling job.
Org-level restrictions. An organization owner can cap the default token to read-only and prevent repositories from overriding it. If your permissions: block looks correct and the call still returns 403, check the org's Actions settings before you debug the YAML.

When a step needs more than the token can ever give, such as pushing to a different repository or triggering a workflow from a workflow (events created with GITHUB_TOKEN generally do not start new runs), use a GitHub App installation token or a fine-grained personal access token stored as a secret. A GitHub App is the better long-term choice because it is not tied to one person's account. The honest drawback: it adds setup (create the app, install it, mint a token in the job), so use it only when permissions: genuinely cannot cover the case.

Takeaway: if the YAML is right and it still 403s, the ceiling is set by the calling workflow or the organization, not by your job.

FAQ

Why does GitHub Actions say "Resource not accessible by integration"?

The GITHUB_TOKEN for that run does not have the permission scope the API endpoint requires. Add the matching scope, such as pull-requests: write, to the job's permissions: block.

Is it safe to set workflow permissions to "read and write" for all workflows?

It works, but it gives every job, including ones that run third-party actions, write access it does not need. Keep the repository default read-only and grant write per job.

Why can't my workflow comment on pull requests from forks?

Fork PRs run with a read-only GITHUB_TOKEN and no secrets by design. Post the comment from a separate workflow_run workflow instead.

Bottom line

If you own the repo and the failing job is a trusted internal one, add a job-level permissions: block with the single scope from the table and stop there. If the failure only happens for forks or Dependabot, restructure so the privileged step runs through workflow_run rather than loosening the token. If you need cross-repository writes or chained workflow triggers, move to a GitHub App token. Treat "read and write for everything" as a last resort, not a fix.

Vercel vs Netlify vs Cloudflare Pages: Which Meter Will Your Frontend Hosting Bill Actually Grow On?

Libme — Sat, 07 Nov 2026 15:14:07 +0000

The headline plan price rarely decides your frontend hosting bill. Three meters do: outbound bandwidth, server-side compute (functions, SSR, image optimization), and per-seat charges. Cloudflare Pages is the safest choice when bandwidth is the risk, Vercel is the smoothest when you run Next.js with SSR and a team, and Netlify sits in between with a model that has changed enough that you should re-read it before committing.

All plan details below are as of November 2026. These vendors revise pricing often, so use this post for the structure of the decision and confirm current numbers on each pricing page.

What is a frontend hosting bill actually made of?

Almost every overage I have seen traces back to one of four line items:

Bandwidth (egress): bytes served from the CDN to browsers. Big images, video, and source maps nobody needed to ship all land here.
Compute: serverless function invocations and duration, SSR rendering, middleware/edge execution, and image optimization requests.
Build minutes and concurrency: how long CI-style builds take and how many can run at once.
Seats: some plans charge per team member who can deploy, even if they only deploy once a month.

A static marketing site is dominated by item 1. A Next.js app with server components and personalised pages is dominated by item 2. A five-person agency with forty small sites is dominated by item 4. Pick the host that is cheapest on your dominant meter, not the one with the nicest free tier.

Takeaway: identify your dominant meter first; comparing plan sticker prices before that is comparing the wrong number.

How do the three differ on each meter?

Meter	Vercel	Netlify	Cloudflare Pages
Bandwidth	Included allowance per plan, metered overage beyond it	Included allowance, metered beyond it (pricing model revised in recent years, check current)	Static asset requests are not metered for bandwidth
Server compute	Metered functions (invocations, duration, memory); strongest Next.js support	Metered functions and edge functions	Pages Functions run on Workers and bill by request/CPU time on the Workers plan
Seats	Paid plans charge per deploying member	Paid plans have team-member pricing	Seats are not the lever; account-level plan
Fit for Next.js	Native, first-party	Supported through an adapter/runtime	Supported via adapters; some features lag or behave differently
Main drawback	Overages and per-seat cost surprise growing teams	Credit/plan model needs reading carefully; costs hard to predict	Smaller feature surface for framework-specific SSR behaviour

The pattern is that Cloudflare monetises compute and add-on products rather than static egress, while Vercel and Netlify meter the delivery itself. That is why a viral image-heavy page can be a non-event on one and a surprise invoice on another.

Takeaway: if your site is mostly static assets and traffic is unpredictable, an unmetered-egress host removes the worst tail risk.

When does SSR or image optimization become the real cost?

This is where I have seen Next.js projects get caught. Rendering a page on every request means every visitor consumes compute, and image optimization means each unique width/format combination can be a billable transformation on hosts that meter it.

Before you migrate or upgrade, find out which routes are actually dynamic. In a Next.js project, the build output lists them:

# After `next build`, the route table marks each route.
# ○ = static, ● = SSG with data, ƒ = dynamic (server-rendered on demand)
npx next build 2>&1 | grep -E '^\s*(○|●|ƒ)'

Every ƒ route is a compute meter. Often a route is dynamic only because it reads cookies() or headers() somewhere in the tree, or because a fetch is uncached. Converting accidental dynamic routes to static or revalidated ones cuts the compute meter on every host, which is a bigger lever than switching providers.

For images, two honest options exist: keep the host's optimizer and cap the variants (a limited deviceSizes list in next.config.js), or pre-generate sizes at build time and serve them as plain static files. The second removes the meter entirely at the cost of a build step you maintain.

Takeaway: audit which routes are dynamic before blaming the host; accidental SSR is the most common avoidable compute cost.

How do I estimate my bill before I commit?

Do not trust intuition; plug your own numbers into a model with the rates from each pricing page. This small script makes the structure explicit. The rates are placeholders you must fill in yourself, deliberately not real prices.

from dataclasses import dataclass

@dataclass
class Plan:
    name: str
    base_monthly: float        # plan fee
    seats_included: int
    per_extra_seat: float
    gb_included: float
    per_extra_gb: float        # 0 if egress is unmetered
    fn_million_included: float
    per_extra_million_fn: float

def monthly_cost(p: Plan, seats: int, gb_out: float, fn_millions: float) -> float:
    extra_seats = max(0, seats - p.seats_included)
    extra_gb = max(0.0, gb_out - p.gb_included)
    extra_fn = max(0.0, fn_millions - p.fn_million_included)
    return (p.base_monthly
            + extra_seats * p.per_extra_seat
            + extra_gb * p.per_extra_gb
            + extra_fn * p.per_extra_million_fn)

# Fill from each vendor's current pricing page. These zeros are placeholders.
plans = [
    Plan("host_a", 0, 1, 0, 0, 0, 0, 0),
    Plan("host_b", 0, 1, 0, 0, 0, 0, 0),
]

# Run three scenarios: today, 3x traffic, and a viral spike month.
scenarios = {"today": (3, 200, 2), "3x": (3, 600, 6), "spike": (3, 5000, 10)}
for p in plans:
    for label, (seats, gb, fn) in scenarios.items():
        print(p.name, label, round(monthly_cost(p, seats, gb, fn), 2))

The scenario that matters is the spike month, not today. A host with a slightly higher base fee and unmetered egress often wins the spike scenario by a wide margin, while a host with generous bundled compute wins the dynamic-app scenario.

Takeaway: model today, 3x, and a spike month; the host that wins only "today" is the one that surprises you later.

Is it worth paying for the platform that matches my framework?

Sometimes. If you run Next.js with server components, ISR and middleware, the first-party host removes a class of "works locally, behaves differently deployed" problems, and that time has real value. If you ship a static Astro, Vite, or Hugo site, you are paying a premium for features you do not use, and an unmetered-egress host is the rational default.

Each option has a real downside. Vercel's per-seat pricing punishes teams where many people occasionally deploy, and its overages can arrive without a hard stop unless you configure spend controls. Netlify's pricing structure has been revised enough that older blog posts (including comparisons from a couple of years ago) are unreliable. Cloudflare Pages gives you the least friction on cost but, for framework-specific SSR features, you may hit behavioural differences from Node-based hosting because the runtime is not Node by default.

If you want the managed Next.js experience and accept the metering, Vercel is the host that handles SSR, ISR and image optimization without extra adapter work. If your risk is egress on a mostly static site, Cloudflare Pages is the one that does not bill you for serving static assets.

Takeaway: pay for framework fit only if you use the framework features that need it.

FAQ

Is Cloudflare Pages really free for unlimited bandwidth?

Static asset requests on Pages are not metered for bandwidth as of November 2026, but Pages Functions run on Workers and are subject to Workers request and CPU limits and pricing. Check the current limits for your plan.

Why is my Vercel bill higher than the plan price?

Usually usage beyond the included allowances (bandwidth, function duration, image optimization) or additional paid seats. Open the usage dashboard and sort by line item to see which one grew.

Can I move a Next.js app off Vercel to avoid costs?

Yes, but test every dynamic feature (middleware, ISR, image optimization) on the target host first, since adapters and runtimes do not always match Vercel's behaviour exactly.

Bottom line

Static or image-heavy sites with unpredictable traffic fit Cloudflare Pages, because egress is the risk and it is not metered there. Next.js teams that use SSR, ISR and middleware heavily will usually spend less engineering time on Vercel, as long as they set spend limits and count seats honestly. Netlify is worth a fresh look if you value its build and form tooling, but read its current pricing model rather than relying on older comparisons. Whatever you choose, audit your dynamic routes first; that fixes more of the bill than any migration.

LLM Returns Invalid JSON Once in a While: Structured Outputs vs Validate-and-Retry

Libme — Fri, 06 Nov 2026 22:31:25 +0000

TL;DR: If json.loads() on model output fails only occasionally, there are three distinct causes: truncation, stray prose or code fences around the JSON, and schema drift (valid JSON, wrong shape). Provider-side structured outputs (constrained decoding) eliminate the second and third for supported schemas; a validate-and-retry loop with a typed model covers everything else, including the first. In production you usually want both.

What does the failure actually look like?

The symptom that sends people to a search engine is usually one of these:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 140 (char 139)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Ticket
priority
  Input should be less than or equal to 4 [type=less_than_equal, ...]

They look alike but have different fixes:

Symptom	Real cause	Right fix
`Expecting value: line 1 column 1`	Model wrapped the JSON in a code fence or added "Here is the JSON:"	Constrained decoding, or extract the outermost braces
`Unterminated string` / `Expecting ',' delimiter` near the end	Output hit `max_tokens` mid-object (truncation)	Raise the limit, shrink the schema, check the stop reason
Parses fine, then your code crashes on a missing key or odd enum value	Schema drift: valid JSON, wrong shape	Typed validation, or a strict schema on the provider side

The first thing to do before changing any prompt: log the raw response and the stop/finish reason for every failure. Most "the model is flaky" reports I have seen turned out to be truncation, which no prompt wording can fix.

Takeaway: classify the failure from the raw text and the stop reason before you touch the prompt.

Step 1: Validate with a typed model instead of `json.loads`

Parsing and validating in one step gives you an error message precise enough to feed back to the model. This uses Pydantic v2 and works with any provider, because call_model is just a function you supply.

from typing import Callable, Literal
from pydantic import BaseModel, Field


class Ticket(BaseModel):
    category: Literal["bug", "billing", "feature_request"]
    priority: int = Field(ge=1, le=4)
    summary: str = Field(min_length=1, max_length=200)


def extract_json_object(raw: str) -> str:
    """Strip code fences / chatter by taking the outermost braces."""
    start, end = raw.find("{"), raw.rfind("}")
    if start == -1 or end <= start:
        raise ValueError("no JSON object found in response")
    return raw[start : end + 1]


def parse_ticket(raw: str) -> Ticket:
    return Ticket.model_validate_json(extract_json_object(raw))

The brace extraction is a blunt tool: it will not rescue truncated output, and it can mis-slice if the model emits two objects. That is fine, because the retry loop below catches those cases.

Step 2: Retry with the error, not with the same prompt

Resending the identical request is a coin flip. Sending the validation error back gives the model something concrete to correct. In Pydantic v2, ValidationError subclasses ValueError, so one except covers both parse and schema failures.

Message = dict[str, str]


def extract_ticket(
    call_model: Callable[[list[Message]], str],
    prompt: str,
    max_attempts: int = 3,
) -> Ticket:
    messages: list[Message] = [{"role": "user", "content": prompt}]
    last_error: ValueError | None = None

    for _ in range(max_attempts):
        raw = call_model(messages)
        try:
            return parse_ticket(raw)
        except ValueError as exc:
            last_error = exc
            messages += [
                {"role": "assistant", "content": raw},
                {
                    "role": "user",
                    "content": (
                        "That response failed validation:\n"
                        f"{exc}\n"
                        "Reply with only a corrected JSON object."
                    ),
                },
            ]

    raise RuntimeError(f"no valid output after {max_attempts} attempts") from last_error

Two details that matter. Cap attempts at 2–3: each retry costs a full round trip and re-sends the growing conversation, so an uncapped loop turns one bad input into a bill spike. And if the failure was truncation, retrying with the same max_tokens will truncate again, so check the stop reason and raise the limit instead of looping.

Takeaway: a retry that includes the exact validation error converges; a retry that repeats the prompt just re-rolls the dice.

When do provider-side structured outputs replace the loop?

Most major providers now offer some form of schema-constrained output: OpenAI's Structured Outputs with a strict JSON schema, tool/function calling with an input schema on Anthropic's API, and grammar- or schema-constrained decoding in local runtimes such as llama.cpp and in libraries like Outlines. The model's token choices are restricted so the output cannot violate the schema's syntax. As of late 2026 the details differ per provider, so check current docs for supported schema features.

What this buys you: no code fences, no prose preamble, no invalid enum values, no missing required keys. If you want this without writing the plumbing yourself, OpenAI's Structured Outputs is the option that guarantees schema adherence on a strict schema, at the cost of being tied to that API.

What it does not buy you:

Truncation is still possible. If generation stops at the token limit, the object is still cut off. Strict mode does not make that safe.
Only a schema subset is supported. Constraints like minLength, maximum, or deeply recursive structures are often unsupported or ignored, so your Field(ge=1, le=4) check may still need to run in your own code.
Syntax is not semantics. A schema-valid priority: 4 can still be the wrong priority. Constrained decoding makes the shape certain, not the content correct.
First-request latency. Some providers compile the schema on first use and cache it, so a rarely used schema can be slower on its first call.

Takeaway: constrained decoding guarantees shape, not truncation-safety and not correctness, so keep your own validation behind it.

Which approach should you use?

Situation	Use
Hosted model, simple flat schema	Provider structured outputs plus Pydantic validation
Provider lacks strict mode, or schema uses unsupported constraints	Validate-and-retry loop
Local model (llama.cpp, Ollama-style runtimes)	Grammar/schema-constrained decoding if your runtime supports it, retry loop as backstop
Long outputs (big arrays, many records)	Split into smaller calls; truncation risk grows with output length

FAQ

Why does the LLM return invalid JSON only sometimes?
Sampling is probabilistic, so a model that follows your format 98% of the time will still occasionally add a code fence, stop mid-object at the token limit, or pick an off-schema value. Intermittent failures are normal without constrained decoding.

Does asking for JSON in the prompt guarantee valid JSON?
No. A prompt instruction raises the odds but guarantees nothing; only schema-constrained decoding restricts the output tokens, and even then truncation can break the object.

How many retries should I allow for LLM JSON parsing?
Two or three. If the third attempt still fails, the cause is usually truncation or an ambiguous schema, and more retries will not help.

Bottom line

If you call a hosted model with a simple schema, turn on the provider's strict structured output and keep a Pydantic model behind it for the constraints the schema cannot express. If you are on a provider or local model without it, the validate-and-retry loop above is enough for most small-team workloads. Either way, log the raw response and stop reason so you can tell truncation from drift. Cap retries, because the loop's failure mode is cost, not correctness.

Docker Build Secrets: Why ARG and COPY Leak Tokens Into Image History, and the BuildKit Fix

Libme — Thu, 05 Nov 2026 18:32:32 +0000

If you pass an npm token, pip index password, or SSH key into docker build with ARG, ENV, or COPY, it can be recovered from the image afterward, even if a later step deletes it. The fix is a BuildKit secret mount (RUN --mount=type=secret), which exposes the secret to one RUN step as a temporary file and never writes it into a layer. Below: how to prove the leak on your own image, the replacement pattern, and the traps that remain.

How do I check whether my image already leaks a secret?

Start with the evidence, not the theory. Take a Dockerfile that looks reasonable:

FROM node:22-slim
WORKDIR /app
ARG NPM_TOKEN
COPY package*.json ./
RUN echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > .npmrc \
 && npm ci \
 && rm .npmrc
COPY . .

The author deleted .npmrc in the same step, so it feels safe. Build it and inspect:

docker build --build-arg NPM_TOKEN=npm_fake_example_token -t leaky .
docker history --no-trunc leaky | grep -i npm_token

The ARG value appears in the recorded command of that layer in the history output, because the expanded RUN instruction is part of the image metadata. Anyone who can pull the image can run docker history or docker inspect on it.

Recent Docker versions also warn at build time with a check named SecretsUsedInArgOrEnv ("Do not use ARG or ENV instructions for sensitive data"). It is name-based, so it catches NPM_TOKEN or PASSWORD but will not catch a secret in a variable called X. Treat it as a smoke alarm, not an audit.

The same applies to COPY .npmrc . followed by RUN rm .npmrc: the COPY layer is a tarball of the file, and deleting it in a later layer only hides it from the merged filesystem. Extract the layers with docker save and the file is there.

Takeaway: deleting a file in a later layer does not remove it from the image; it only masks it.

What does the BuildKit secret mount do differently?

A secret mount makes a file (or env value) available only while one RUN instruction executes. It is mounted on a tmpfs-style path, not committed into the layer, and not recorded as a build argument.

# syntax=docker/dockerfile:1
FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci
COPY . .

Build by pointing the id at a source on the host:

docker build --secret id=npmrc,src=$HOME/.npmrc -t safe .

If the token is in an environment variable instead of a file, BuildKit can read it from the environment of the build command:

export NPM_TOKEN=npm_fake_example_token
docker build --secret id=NPM_TOKEN,env=NPM_TOKEN -t safe .

RUN --mount=type=secret,id=NPM_TOKEN \
    echo "//registry.npmjs.org/:_authToken=$(cat /run/secrets/NPM_TOKEN)" > /tmp/.npmrc \
 && NPM_CONFIG_USERCONFIG=/tmp/.npmrc npm ci \
 && rm /tmp/.npmrc

By default a secret mounted without target appears at /run/secrets/<id>. Note the second form writes a temp file inside the layer filesystem, but it is removed within the same RUN, and the token was never in the instruction text, so neither docker history nor the committed layer contains it. If you can use the first form with a real file at target, prefer it: there is nothing to clean up.

Takeaway: a secret mount keeps the value out of both the instruction text and the layer contents, which is the two places ARG and COPY leak.

Which method leaks what?

Method	In `docker history`	In a layer tarball	Survives to final image	Use for secrets?
`ARG` + `--build-arg`	Yes, when used in `RUN`	Possibly, if written to a file	Metadata yes	No
`ENV`	Yes	n/a	Yes, and in running containers	No
`COPY` then `rm`	No	Yes	Masked, recoverable	No
Multi-stage, secret only in builder stage	In builder only	In builder only	No, if final stage does not copy it	Acceptable, with caveats
`RUN --mount=type=secret`	No	No	No	Yes
`RUN --mount=type=ssh`	No	No	No	Yes, for git over SSH

The multi-stage row deserves the caveat: the final image is clean, but the builder stage's layers still exist in your local build cache, and if you export cache with --cache-to to a registry, those layers can go with it. Which is why "multi-stage" alone is not a secrets strategy.

What about private git dependencies and CI?

For git clone or pip install git+ssh://... from a private repo, use the SSH mount rather than copying a key:

# syntax=docker/dockerfile:1
RUN --mount=type=ssh \
    pip install "git+ssh://git@github.com/your-org/private-lib.git"

docker build --ssh default -t app .

This forwards your local SSH agent for that step only, so no private key lands in a layer. You will also need github.com in known_hosts inside the build, which is the usual first failure: Host key verification failed.

In GitHub Actions with docker/build-push-action, the secrets input takes the same id=value pairs:

- uses: docker/build-push-action@v6
  with:
    context: .
    push: true
    tags: ghcr.io/your-org/app:latest
    secrets: |
      npmrc=${{ secrets.NPMRC }}

As of November 2026 this works with BuildKit-based builds, which is the default builder in current Docker releases. Pin the action to the major version you have tested.

Takeaway: the pattern is identical locally and in CI; only the source of the secret changes.

What still goes wrong after you switch?

A secret echoed into build output. If a step fails verbosely (npm ci with debug logging, pip -vvv), the token can appear in CI logs. Mask it in your CI system and keep verbosity off.
Secrets written to a path the tool caches. Some package managers copy credentials into a cache directory inside the layer. Check with docker run --rm safe sh -c 'find / -name ".npmrc" 2>/dev/null' after the build.
Old images are already published. Switching Dockerfiles does not un-leak what is in your registry. If an image with a real token was ever pushed, rotate the token first; deleting the tag is secondary.
Inspecting tools have limits. dive is useful for browsing layer contents interactively, but it is a manual check and will not scan for credentials. A dedicated secret scanner run against the saved image tarball is a better gate in CI.

If you want automated detection rather than eyeballing layers, TruffleHog is the open-source scanner that can check container images for verified credentials; its limitation is that it finds what looks like a known credential format, so an internal password with no recognizable pattern slips through.

FAQ

Does deleting a file in a later Dockerfile step remove it from the image?
No. Each instruction creates a new layer, and a delete only adds a "whiteout" marker in the new layer. The earlier layer still contains the file and can be extracted from the image.

Is multi-stage build enough to keep secrets out of the final image?
It keeps them out of the final image's layers if the final stage never copies them, but the builder layers remain in your build cache and possibly in exported cache. Use a secret mount so the value never enters a layer at all.

How do I use a secret in docker build without BuildKit?
You can't do it safely with the legacy builder. Enable BuildKit (the default in current Docker Engine and Docker Desktop) and use --secret with RUN --mount=type=secret.

Bottom line

If a credential is needed only to install dependencies during the build, use RUN --mount=type=secret for it, and --mount=type=ssh for private git over SSH. Stop using ARG and ENV for anything you would not paste into a public chat. If an image with a real secret was ever pushed, rotate the secret before you fix the Dockerfile.

ENOSPC: System Limit for Number of File Watchers Reached — Raise the Limit, Shrink the Watch, or Poll?

Libme — Wed, 04 Nov 2026 15:13:09 +0000

If your dev server dies with Error: ENOSPC: System limit for number of file watchers reached, it has nothing to do with disk space. You've hit the Linux kernel's inotify watch limit. You have three fixes: raise fs.inotify.max_user_watches on the host, make your tools watch fewer directories, or switch to polling when inotify can't work at all (some Docker bind mounts, network drives).

The right choice depends on which of those situations you're in, and the order matters. Raising the limit is the fastest fix, but it hides the real problem if a tool is watching node_modules for no reason.

What does ENOSPC mean when there's plenty of disk space?

ENOSPC is "no space left on device", but the kernel reuses it for inotify. The message usually looks like this:

Error: ENOSPC: System limit for number of file watchers reached, watch '/home/dev/app/src/components'
    at FSWatcher.<computed> (node:internal/fs/watchers:247:19)

Two different limits produce it:

Limit	What it caps	Typical symptom
`fs.inotify.max_user_watches`	Total watches per user (roughly one per watched directory)	`...System limit for number of file watchers reached, watch '<path>'`
`fs.inotify.max_user_instances`	Number of inotify handles per user	Same ENOSPC, but no path, or `inotify_init` failing

Linux inotify watches directories, not individual files, so the cost scales with directory count. A monorepo with several thousand folders, plus a bundler, a test runner, a TypeScript server and an editor all watching the same tree, can use up the limit quickly. As of kernel 5.11 the default max_user_watches scales with system RAM (up to about a million); older kernels and many distro configs default to 8192, which is where most people hit it.

Takeaway: ENOSPC from a watch call is an inotify quota error, and the quota is per user, shared by every tool you have running.

How do I see who is using all the watches?

Before you change anything, check the current limits and find the biggest consumer:

cat /proc/sys/fs/inotify/max_user_watches
cat /proc/sys/fs/inotify/max_user_instances

# Watches per process (run with sudo to see other users' processes)
for fd in $(find /proc/*/fd -lname 'anon_inode:inotify' 2>/dev/null); do
  pid=$(echo "$fd" | cut -d/ -f3)
  n=$(grep -c '^inotify' "/proc/$pid/fdinfo/$(basename "$fd")" 2>/dev/null)
  echo "$n $pid $(tr '\0' ' ' < "/proc/$pid/cmdline" | cut -c1-80)"
done | sort -rn | head

In my experience the top entry is rarely the thing you expected. An editor's file watcher, a second dev server you forgot about, or a Jest watch mode running with the whole repo as its root are common culprits. If one process owns most of the watches and it's watching node_modules or .git, fix that first (next sections) rather than raising the limit.

To estimate what a single tool needs, count directories it would actually watch:

find . -type d -not -path '*/node_modules/*' -not -path '*/.git/*' | wc -l

Takeaway: one process typically owns most of the watches, and finding it takes thirty seconds.

How do I raise the inotify limit permanently?

This is the standard fix on a Linux dev machine or CI runner. A one-off change:

sudo sysctl fs.inotify.max_user_watches=524288
sudo sysctl fs.inotify.max_user_instances=512

To persist across reboots, put it in a drop-in file:

printf 'fs.inotify.max_user_watches=524288\nfs.inotify.max_user_instances=512\n' \
  | sudo tee /etc/sysctl.d/99-inotify.conf
sudo sysctl --system

The numbers above are common community values, not magic. Each watch costs kernel memory (on the order of a kilobyte or two on 64-bit systems), and that memory is unswappable, so setting the limit to the maximum on a small VM is not free. Size it to your tree plus headroom.

Where you apply it matters. The limit lives in the kernel, so:

On a Linux host running containers, set it on the host. Containers share the host kernel.
On Docker Desktop for Mac or Windows, the "host" is the Linux VM Docker runs, not your laptop OS.
In WSL2, set it inside the WSL distribution.
In managed CI, you often can't change it, which pushes you toward the next two options.

Takeaway: raise max_user_watches on the machine whose kernel is actually doing the watching, and write it to /etc/sysctl.d/ so it survives a reboot.

How do I make my tools watch fewer directories?

Raising the limit treats the symptom. Excluding directories that never change at dev time treats the cause. Settings for the common tools (as of late 2026, check your version's docs):

Tool	Setting
Vite	`server.watch.ignored`
webpack / webpack-dev-server	`watchOptions.ignored`
Next.js	Uses webpack/Turbopack watchers; keep generated and output directories out of the project root where possible
Jest	`watchPathIgnorePatterns`, or `--no-watchman` to avoid the Watchman crawler
VS Code	`files.watcherExclude`
tsc `--watch`	`watchOptions.excludeDirectories` in `tsconfig.json`

For example, a Vite config that stops watching build output and a vendored directory:

// vite.config.ts
import { defineConfig } from 'vite';

export default defineConfig({
  server: {
    watch: {
      ignored: ['**/dist/**', '**/.turbo/**', '**/vendor/**'],
    },
  },
});

And for the TypeScript watcher:

{
  "watchOptions": {
    "excludeDirectories": ["**/node_modules", "**/dist"]
  }
}

A caveat from experience: ignoring a directory means edits there will not trigger a rebuild. If you symlink a local package into node_modules for development, excluding node_modules will make that package's changes invisible, so check before you ignore.

If you run several tools that each crawl the repo, Watchman is the one that gives you a single shared watcher service instead of one crawler per tool, which Jest and Metro can use. Its drawback is that on Linux it still uses inotify underneath, so it reduces duplication but does not remove the need for a sane limit, and it adds a daemon to maintain.

Takeaway: if a tool is watching node_modules or build output, exclude them in its config before you touch the kernel setting.

When should I switch to polling instead?

Polling stat-checks files on a timer, so it uses no inotify watches at all. It trades CPU and latency for reliability. Use it when inotify genuinely can't see changes, not as a reflex:

Bind mounts from macOS or Windows into a Linux container often don't deliver filesystem events reliably, so hot reload silently never fires.
Network filesystems (NFS, SMB) generally don't produce inotify events for changes made on other machines.
Locked-down CI where you can't change sysctl values.

Common switches:

# chokidar-based tools (older webpack setups, nodemon alternatives)
CHOKIDAR_USEPOLLING=true npm run dev

# Next.js (webpack watcher)
WATCHPACK_POLLING=true npm run dev

// vite.config.ts
export default defineConfig({
  server: { watch: { usePolling: true, interval: 300 } },
});

Polling a large tree at a short interval will visibly burn CPU. Pair it with the ignore lists above so you only poll source directories, and keep the interval as long as you can tolerate.

Takeaway: polling is the fallback for places where inotify events never arrive, and it needs an ignore list to stay cheap.

FAQ

How do I fix "ENOSPC: System limit for number of file watchers reached"?
Run sudo sysctl fs.inotify.max_user_watches=524288 and persist it in /etc/sysctl.d/99-inotify.conf. If a tool is watching node_modules or build output, exclude those paths too.

Does ENOSPC in Node mean my disk is full?
Not when the message mentions "file watchers" or a watch call. That is the inotify watch or instance limit. A genuinely full disk shows up in write operations, and df -h will confirm it.

Do I need to set the inotify limit inside the Docker container?
No, the container shares the host kernel, so set it on the Linux host or in the Docker Desktop VM. Containers typically can't change it with their own sysctl settings.

Bottom line

On a Linux machine you control, raise fs.inotify.max_user_watches via /etc/sysctl.d/, but first spend a minute finding which process holds the watches and excluding node_modules and build output from it. If you're on a bind mount or network drive where events never arrive, use polling with a tight ignore list. If you can't change kernel settings, shrinking the watched set is your only real lever.

Docker Desktop Alternatives on Mac: OrbStack vs Colima vs Podman Desktop

Libme — Tue, 03 Nov 2026 22:15:35 +0000

If you want the lowest-friction swap, OrbStack is the closest drop-in for Docker Desktop on a Mac, with the licensing caveat that commercial use is paid. If you want free and open source with a CLI-first workflow, Colima keeps your existing docker and docker compose commands working. Podman Desktop is the right pick when rootless, daemonless containers are a requirement rather than a preference.

This post is for the moment you are about to pay for Docker Desktop, or a licensing email has landed in your inbox, and you need to know what actually changes when you leave. Details are as of November 2026; licensing terms and features move, so check each vendor's page before you commit a team.

Why look for an alternative at all?

Three reasons come up repeatedly: licensing, resource usage, and file-sharing speed. Docker Desktop's subscription requirement applies to larger companies (Docker's published terms tie it to company size and revenue), while personal use, education, and small businesses are exempt. If you are a solo developer, the licence probably does not apply to you, and you should switch only for the other two reasons.

On macOS every option here runs Linux containers inside a lightweight VM. The differences are which VM layer they use, how they share files with the host, and how much of the Docker developer experience they reproduce.

Takeaway: on a Mac, "which Docker alternative" is really "which Linux VM manager and which file-sharing layer".

How do the three compare?

	OrbStack	Colima	Podman Desktop
Licence model	Proprietary, free for personal use, paid for commercial (check current terms)	Open source (MIT)	Open source (Apache 2.0)
Engine	Docker-compatible engine	Docker (default) or containerd, via Lima	Podman, daemonless
GUI	Native Mac app	None (CLI)	Electron app
`docker` CLI compatibility	Works out of the box	Works via a Docker context	Needs the socket compatibility mode or `podman` alias
Compose	`docker compose`	`docker compose`	`podman compose` or a Docker-compatible socket
Local Kubernetes	Built in, optional	`colima start --kubernetes`	Via kind/minikube integrations
Rootless by default	No	No	Yes
Platform	macOS only	macOS and Linux	macOS, Windows, Linux

Two things the table hides. First, OrbStack is closed source, so you are trusting one vendor instead of a community project. Second, Podman is not Docker with a different logo: most commands translate, but anything that talks directly to /var/run/docker.sock needs configuration.

Takeaway: the further you move from the Docker daemon, the more you pay in compatibility work.

How do you set up Colima without breaking existing scripts?

Colima is the option I would try first if you are comfortable in a terminal, because your existing commands stay the same.

brew install colima docker docker-compose

# vz = Apple's Virtualization framework; virtiofs is the faster mount type
colima start --cpu 4 --memory 8 --disk 60 --vm-type vz --mount-type virtiofs

docker context ls          # a "colima" context should now be active
docker run --rm hello-world

Plan on running colima start after every reboot, or register it as a Homebrew service with brew services start colima. If you forget, every docker command fails with a connection error against the socket, which is a confusing first-day symptom.

The most common breakage is tooling that hardcodes the default socket path. Anything that does not read your Docker context needs to be pointed at Colima's socket explicitly:

export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock"
# Testcontainers has its own override for the socket it mounts into containers
export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock

Without those, the symptom I would look for is a test suite that works under Docker Desktop and fails at startup with "Could not find a valid Docker environment".

The honest drawback: Colima has no GUI and fewer guardrails. The VM's CPU, memory, and disk are fixed at start time, so if your builds run out of memory you restart the VM with new flags rather than dragging a slider.

For a developer who wants to keep the Docker CLI and pay nothing, Colima is the option that changes the least while removing the subscription question.

Takeaway: Colima keeps your commands, and the only migration cost is socket paths in tools that bypass your Docker context.

When is OrbStack worth it?

OrbStack's pitch is speed and a native feel: quick startup, lower idle resource use, and fast file sharing, with a Mac-native app instead of an Electron one. In my experience the practical difference is largest on bind-mount-heavy workloads, such as a Node project with a large node_modules mounted into a container. Benchmark it on your own project before believing anyone's numbers, including mine; I am not quoting figures because they vary heavily with the repo and the Mac.

If you want a fast, polished Docker Desktop replacement and your employer will pay for the commercial licence, OrbStack is the one that feels least like a compromise. Its limits are real: macOS only (so your team's Linux and Windows developers need something else), closed source, and a licence you have to track.

Takeaway: OrbStack buys polish and speed, and the price is a vendor dependency and a licence check for commercial use.

When does Podman Desktop make sense?

Choose Podman when the security model is the point: no root daemon, rootless containers by default, and a CLI that mirrors Docker closely enough that alias docker=podman covers most daily use.

brew install podman
podman machine init --cpus 4 --memory 8192
podman machine start
podman run --rm docker.io/library/hello-world

Note the fully qualified image name. Podman may prompt you to pick a registry for short names, which breaks non-interactive scripts until you configure unqualified-search-registries. Rootless mode also means you cannot bind ports below 1024 on Linux hosts without extra setup, and docker-compose-based projects sometimes need adjustment. If your team ships to environments where Podman is already the runtime, the parity benefit is genuine.

Takeaway: pick Podman for rootless and runtime parity, not as a licensing workaround, because it is the biggest behavioural change of the three.

How should you decide?

Your situation	Pick
Solo developer or small business under Docker's licence threshold	Stay on Docker Desktop, or Colima if you prefer CLI
Company that needs to cut licence cost, team comfortable in a terminal	Colima
Company that will pay for polish and speed, macOS-only team	OrbStack
Security or compliance requires rootless, or production runs Podman	Podman Desktop
Mixed Mac, Windows, Linux team needing one tool	Podman Desktop or Docker Desktop

FAQ

Is Colima a drop-in replacement for Docker Desktop?

For the docker and docker compose CLI, yes, since Colima runs a real Docker engine in a VM and exposes it through a Docker context. It is not a drop-in for the GUI, Docker extensions, or tools that hardcode the default socket path.

Is Docker Desktop free?

Docker's terms make it free for personal use, education, open source, and small businesses, and paid for larger companies. Check Docker's current subscription terms for the exact thresholds, since they are the ones that apply to you.

Is OrbStack faster than Docker Desktop?

It is commonly reported as faster at startup and file sharing, but the gain depends on your workload, so run your own build and test loop under both before deciding.

Bottom line

Solo developers under the licence threshold have no urgent reason to leave Docker Desktop. If you do want to leave, Colima is the safest default because your commands, Compose files, and mental model all carry over. If you want the smoothest Mac experience and someone else pays, OrbStack is the one to try. Choose Podman Desktop only when rootless containers or Podman parity in production is the actual requirement.

pip "externally-managed-environment" Error: venv, pipx, or uv?

Libme — Mon, 02 Nov 2026 22:16:10 +0000

If pip install fails with error: externally-managed-environment, your operating system's package manager owns that Python, and pip is refusing to write into it (PEP 668). The fix is almost never --break-system-packages: use a virtual environment for project dependencies, pipx or uv tool for command-line applications, and reserve the override flag for throwaway containers.

What does the error actually say, and why does it happen?

On a recent Debian, Ubuntu, or Homebrew-installed Python, this is what you get:

$ pip install requests
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    ...
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at your own risk, by passing the --break-system-packages flag.

The mechanism is a marker file. Distributions that ship PEP 668 support place a file named EXTERNALLY-MANAGED in the interpreter's standard library directory, and pip checks for it before any install into that environment. You can see it yourself:

python3 -c "import sysconfig, pathlib; p = pathlib.Path(sysconfig.get_path('stdlib')) / 'EXTERNALLY-MANAGED'; print(p, p.exists())"

The reason it exists is real: apt and pip both write into the same site-packages, and when pip upgrades a library that an OS tool depends on, the OS tool can break in ways that are miserable to diagnose. As of late 2026 this behavior is the default on current Debian and Ubuntu releases, and Homebrew's Python applies it too. It is not a pip bug.

Takeaway: the error means "this interpreter belongs to your OS", and the right response is to stop installing into it, not to silence the check.

Which dead ends should you skip?

These are the ones I see suggested most often, and why they cost you later:

sudo pip install ... does not bypass the check, and if it did, you would be writing into the exact directory the protection exists for.
Deleting the EXTERNALLY-MANAGED file works until the next Python package update puts it back, and in the meantime you have turned off the guard for every user on the machine.
pip install --user is blocked by the same marker on most setups, so it does not help.
--break-system-packages works, and on a disposable machine that is fine. On a laptop or server you care about, it reintroduces the exact apt-versus-pip conflicts PEP 668 prevents.

Takeaway: every shortcut here trades a one-minute fix now for an unexplainable breakage of an OS tool later.

Which tool fits which job?

The right choice depends on what you are installing, not on personal taste.

Situation	Best fit	Why
Dependencies for a project you are developing	`venv` (or `uv venv`)	One isolated environment per project, reproducible from a requirements file
A CLI tool you just want on your PATH (e.g. a formatter, `httpie`)	`pipx` or `uv tool install`	Each app gets its own hidden environment, the command lands on PATH
Many projects, slow installs, lockfile wanted	`uv`	Fast resolver and installer, handles Python versions too
Docker image or CI job that is thrown away	`venv`, or the override flag	The OS-protection argument is weaker in a disposable root filesystem
A library packaged by your distro	`apt install python3-<name>`	Managed and patched by the OS, but often older

Takeaway: libraries go in a project venv, applications go in an isolated tool environment, and neither goes in the system interpreter.

How do you set up a venv correctly (including the Debian gotcha)?

The standard library route needs no extra tools:

python3 -m venv .venv
source .venv/bin/activate
pip install requests

On Debian and Ubuntu you may hit a second error first: The virtual environment was not created successfully because ensurepip is not available. Debian splits venv into its own package, so run sudo apt install python3-venv and retry.

Do not skip source .venv/bin/activate. If you forget, pip resolves to the system one and you are back at the original error. When in doubt, call the venv's interpreter explicitly: .venv/bin/python -m pip install requests.

If you want the managed version of this workflow, uv is the tool that creates the environment and installs from a requirements file in one fast step:

uv venv
uv pip install -r requirements.txt

A genuine drawback: uv is a separate binary you have to install and trust, and its uv pip interface follows pip closely but not identically, so edge cases in older requirements files can resolve differently than they did under pip. Test before you swap it into CI.

Takeaway: the venv is only protecting you if it is active or invoked by path; most "it still fails" reports are an unactivated shell.

How do you install command-line tools without polluting anything?

For applications, pipx creates one isolated environment per tool and exposes only its executables:

sudo apt install pipx   # or: brew install pipx
pipx ensurepath
pipx install httpie

If you want the one that handles isolated tool installs without a separate package from your distro, uv tool install httpie does the same job with the same one-environment-per-app model.

The limitation of both: they are for applications. If you pipx install a library and then import it in your own script, it will not be importable, because the library lives in its hidden environment. That is by design, and it is the most common confusion.

Takeaway: if you will type its name in a terminal, use pipx or uv tool; if you will import it, use a venv.

When is `--break-system-packages` acceptable?

In a Dockerfile or CI runner that is rebuilt from scratch every time, there is no OS tool left to protect, and a venv adds a layer for little benefit. Setting ENV PIP_BREAK_SYSTEM_PACKAGES=1 in the image is a reasonable, explicit choice there. Even so, a venv in the image keeps the same code path you use locally, which makes "works on my machine" bugs rarer. Choose the override flag knowingly, not because a forum answer said so.

Takeaway: the override flag is a tool for disposable filesystems, not for your daily laptop.

FAQ

How do I fix "error: externally-managed-environment" on Ubuntu or Debian?

Create a virtual environment with python3 -m venv .venv, activate it, and run pip inside it. If venv creation fails, install python3-venv with apt first.

Is `pip install --break-system-packages` safe?

It is safe on a throwaway container or CI runner. On a machine where apt manages Python tools, it can overwrite files that OS utilities depend on and cause breakage that is hard to trace.

Should I use pipx or uv to install Python command-line tools?

Either works: both give each application its own isolated environment and put its command on your PATH. Choose uv tool if you already use uv for projects, otherwise pipx is available from most package managers.

Bottom line

Use a venv (or uv venv) for anything you import, pipx or uv tool install for anything you run from a terminal, and apt install python3-<name> only when you want the distro-maintained version. Treat --break-system-packages as a Docker-and-CI setting. If you are on a team, put the one-line environment setup in the README so nobody rediscovers this error on day one.

pip "externally-managed-environment" Error: venv, pipx, or uv?

Libme — Sun, 01 Nov 2026 17:19:18 +0000

What does the error actually say, and why does it happen?

On a recent Debian, Ubuntu, or Homebrew-installed Python, this is what you get:

$ pip install requests
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    ...
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at your own risk, by passing the --break-system-packages flag.

python3 -c "import sysconfig, pathlib; p = pathlib.Path(sysconfig.get_path('stdlib')) / 'EXTERNALLY-MANAGED'; print(p, p.exists())"

Takeaway: the error means "this interpreter belongs to your OS", and the right response is to stop installing into it, not to silence the check.

Which dead ends should you skip?

These are the ones I see suggested most often, and why they cost you later:

sudo pip install ... does not bypass the check, and if it did, you would be writing into the exact directory the protection exists for.
Deleting the EXTERNALLY-MANAGED file works until the next Python package update puts it back, and in the meantime you have turned off the guard for every user on the machine.
pip install --user is blocked by the same marker on most setups, so it does not help.
--break-system-packages works, and on a disposable machine that is fine. On a laptop or server you care about, it reintroduces the exact apt-versus-pip conflicts PEP 668 prevents.

Takeaway: every shortcut here trades a one-minute fix now for an unexplainable breakage of an OS tool later.

Which tool fits which job?

The right choice depends on what you are installing, not on personal taste.

Situation	Best fit	Why
Dependencies for a project you are developing	`venv` (or `uv venv`)	One isolated environment per project, reproducible from a requirements file
A CLI tool you just want on your PATH (e.g. a formatter, `httpie`)	`pipx` or `uv tool install`	Each app gets its own hidden environment, the command lands on PATH
Many projects, slow installs, lockfile wanted	`uv`	Fast resolver and installer, handles Python versions too
Docker image or CI job that is thrown away	`venv`, or the override flag	The OS-protection argument is weaker in a disposable root filesystem
A library packaged by your distro	`apt install python3-<name>`	Managed and patched by the OS, but often older

Takeaway: libraries go in a project venv, applications go in an isolated tool environment, and neither goes in the system interpreter.

How do you set up a venv correctly (including the Debian gotcha)?

The standard library route needs no extra tools:

python3 -m venv .venv
source .venv/bin/activate
pip install requests

If you want the managed version of this workflow, uv is the tool that creates the environment and installs from a requirements file in one fast step:

uv venv
uv pip install -r requirements.txt

Takeaway: the venv is only protecting you if it is active or invoked by path; most "it still fails" reports are an unactivated shell.

How do you install command-line tools without polluting anything?

For applications, pipx creates one isolated environment per tool and exposes only its executables:

sudo apt install pipx   # or: brew install pipx
pipx ensurepath
pipx install httpie

If you want the one that handles isolated tool installs without a separate package from your distro, uv tool install httpie does the same job with the same one-environment-per-app model.

Takeaway: if you will type its name in a terminal, use pipx or uv tool; if you will import it, use a venv.

When is `--break-system-packages` acceptable?

Takeaway: the override flag is a tool for disposable filesystems, not for your daily laptop.

FAQ

How do I fix "error: externally-managed-environment" on Ubuntu or Debian?

Create a virtual environment with python3 -m venv .venv, activate it, and run pip inside it. If venv creation fails, install python3-venv with apt first.

Is `pip install --break-system-packages` safe?

It is safe on a throwaway container or CI runner. On a machine where apt manages Python tools, it can overwrite files that OS utilities depend on and cause breakage that is hard to trace.

Should I use pipx or uv to install Python command-line tools?

Bottom line

Git Hooks Not Running for Teammates? Husky vs Lefthook vs pre-commit

Libme — Sat, 31 Oct 2026 17:04:47 +0000

Git hooks don't run for teammates because .git/hooks/ is never committed, so every clone starts with no hooks at all. Husky, lefthook, and pre-commit all solve this the same way: the hook definitions live in the repo, and an install step points Git at them. Pick husky if your repo is already a Node project, lefthook if you want one fast binary with no runtime dependency, and pre-commit if your team is Python-heavy or wants a large catalog of ready-made hooks.

Why do hooks work on my machine but not on a teammate's?

The symptom is boring and easy to miss: a teammate pushes code that your pre-commit lint would have rejected, and nobody sees an error, because nothing ran. There is no failure to find.

Git only executes hooks from the directory in core.hooksPath, which defaults to .git/hooks. That directory is inside .git, so it is not part of the repository content. Anything you put there stays on your machine. You can check what a clone is actually using:

git config --get core.hooksPath   # empty output means the default .git/hooks
ls .git/hooks | grep -v '\.sample$'

If the second command prints nothing, no hooks are installed in that clone. A fresh clone always looks like this.

The first fix people try is committing a scripts/hooks/ folder and asking everyone to symlink it. That works until someone forgets, which is the whole problem. The tools below automate the one step that humans skip.

Takeaway: hooks are local state, so any hook setup that depends on a person remembering to run a command will eventually be missing on someone's machine.

How do husky, lefthook, and pre-commit install themselves?

All three need an install step to run once per clone. They differ in how automatic they can make it.

	Husky	Lefthook	pre-commit
Runtime	Node	Single Go binary	Python
Config	Shell scripts in `.husky/`	`lefthook.yml`	`.pre-commit-config.yaml`
Auto-install trigger	`prepare` script in `package.json`	`lefthook install` (a `prepare` script or a manual step)	`pre-commit install` (manual or a setup script)
Staged-files filtering	Add a tool such as lint-staged	Built in (`{staged_files}`, `glob`)	Built in (hooks receive matching files)
Parallel execution	Not built in	Built in (`parallel: true`)	Not by default for different hooks; it runs hooks serially
Hook catalog	Bring your own commands	Bring your own commands	Large community catalog of hook repos

Husky's trick is that npm install runs the prepare script automatically. As of its v9 line, setup is npx husky init, which creates a .husky/ directory and a prepare entry that runs husky. That command sets core.hooksPath to a path inside .husky/.

{
  "scripts": {
    "prepare": "husky"
  }
}

# .husky/pre-commit
npx lint-staged

Lefthook looks like this:

# lefthook.yml
pre-commit:
  parallel: true
  commands:
    lint:
      glob: "*.{js,ts,tsx}"
      run: npx eslint {staged_files}
    format-check:
      glob: "*.{js,ts,tsx,json,md}"
      run: npx prettier --check {staged_files}

And pre-commit:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.6.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-merge-conflict

Check the rev against the project's latest tag before you copy it; pre-commit autoupdate rewrites it for you.

Takeaway: the real difference between these tools is not the hook syntax, it's what triggers the install on a fresh clone.

What goes wrong even after you adopt one?

These are the failure modes I'd check first when "we have husky, but it still isn't running."

The install step never ran. Husky depends on prepare. If someone installs with npm ci --ignore-scripts, or a package manager is configured to skip lifecycle scripts, prepare doesn't run and no hooks get wired up. The same applies to Docker images and CI, where this is usually what you want, but it also applies to a laptop with a hardened npm config. Verify with git config --get core.hooksPath.

Another tool already owns core.hooksPath. pre-commit refuses to install when it's set, with a message along the lines of "Cowardly refusing to install hooks with core.hooksPath set." If you're migrating from husky to pre-commit, unset it first:

git config --unset-all core.hooksPath
pre-commit install

Git GUIs and PATH problems. A hook that calls npx or node can fail under a GUI client that launches with a different PATH than your shell, especially with version managers like nvm. Lefthook's single binary sidesteps part of this, though the commands it runs still need their own tools on PATH.

Someone passed --no-verify. git commit --no-verify skips pre-commit and commit-msg hooks by design. This is not a bug and you can't prevent it locally.

Takeaway: if hooks stop running, check core.hooksPath and whether the install script executed, in that order, before blaming the hook tool.

Should hooks be the only enforcement?

No. Hooks are a convenience for fast feedback, not a gate. Anyone can skip them, and a contributor on a fork has never run your install step. Run the same checks in CI so the merge button is the actual enforcement.

pre-commit makes this easy: pre-commit run --all-files in a CI job runs the same hook set you run locally. With husky or lefthook, call the same underlying commands (lint, format check) directly from your CI workflow rather than invoking the hook runner. Keep the local hook fast, ideally only the staged files, and let CI do the slow, complete checks.

A drawback worth stating for each: husky needs a Node toolchain and adds nothing for non-JS repos; lefthook has a smaller ecosystem of shared recipes, so you write more commands yourself; pre-commit creates its own isolated environments for hooks, which makes the first run slow and adds cache management to CI.

Takeaway: treat local hooks as a speed-up and CI as the rule, because only CI applies to everyone.

FAQ

Why are my git hooks not running after cloning a repo?
Hooks live in .git/hooks, which is not versioned, so a fresh clone has none. You need a tool or script that installs them, such as npm install triggering husky's prepare, lefthook install, or pre-commit install.

How do I check whether a hook is installed?
Run git config --get core.hooksPath and list the target directory (.git/hooks if the config is empty). A hook file must exist there without the .sample suffix and be executable.

Can I enforce git hooks for everyone on the team?
Not locally, since --no-verify and skipped install steps bypass them. Run the same checks in CI and require them to pass before merging.

Bottom line

If your repo already has a package.json, husky plus a staged-files runner is the least friction, because npm install wires it up for free. If you want parallel hooks and no language runtime, lefthook is the one to try. If your team writes Python or wants ready-made hooks for YAML, secrets, and merge conflicts, pre-commit is the strongest fit. Whichever you choose, mirror the checks in CI.

GitHub Actions Cache Never Hits on Pull Requests: Branch Scope, Immutable Keys, and restore-keys

Libme — Fri, 30 Oct 2026 21:37:17 +0000

If actions/cache reports "Cache not found" on every pull request even though you can see caches in the repo's Actions tab, the cause is almost always one of three things: the cache was saved on a branch your PR cannot read, the key is identical to an existing one (caches are immutable, so the new content is never saved), or your restore-keys prefix does not match anything. All three are fixable in a few lines of YAML, and you can diagnose which one you have from the job log alone.

What does the failure actually look like?

The symptoms I see most often in logs (wording as of late 2026; the exact text may change between releases):

Cache not found for input keys: Linux-npm-3f1a9c..., Linux-npm-

and, in the post-job step:

Warning: Failed to save: Unable to reserve cache with key Linux-npm-3f1a9c..., another job may be creating this cache.

The first line means a restore found nothing. The second means a save was refused because that exact key already exists. People often see both in the same repo and conclude the feature is flaky. It is not. It follows three deterministic rules.

Takeaway: a cache miss is never random; it is one of three rules, and the log tells you which.

Why can't my PR see the cache that main built?

Caches are scoped to a branch (more precisely, to a git ref). According to GitHub's documentation, a workflow run can restore caches created on its own ref, on the base branch of a pull request, and on the default branch. It cannot read caches created by a sibling branch or a different PR.

That has two practical consequences:

If your workflow only runs on pull_request and never on pushes to main, then nothing ever populates the default-branch cache. Every PR starts cold, forever, and each PR writes a cache that no other PR can read.
Two feature branches never share caches with each other. They both fall back to main's cache, if one exists.

The fix is to run the same cache-saving job on pushes to the default branch:

on:
  push:
    branches: [main]
  pull_request:

A related trap: pull requests from forks run with a restricted token, and the cache they write is scoped to the PR merge ref, so don't expect fork PRs to warm anything for others.

Takeaway: if main never runs the cache step, no PR can ever get a warm start from it.

Why does the cache never update when my lockfile changes... or doesn't?

Caches are immutable. Once a key is saved, it cannot be overwritten. This bites in two opposite ways.

Key too static. A key like deps-${{ runner.os }} is saved once, then every later run restores it (an exact hit) and the post-step refuses to save new content. Your cache slowly goes stale and installs get slower, while the log says "Cache hit".

Key too volatile. A key containing ${{ github.sha }} is unique on every commit, so every run is a miss on the primary key and writes a fresh multi-hundred-MB entry. That floods the repository's cache quota and evicts the useful entries.

The standard pattern is a key derived from the file that determines the content, with a prefix fallback:

- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-

When the lockfile changes, the primary key misses, the prefix in restore-keys restores the most recent older cache, npm ci fills in the difference, and the post-step saves a new entry under the new key.

Takeaway: hash the lockfile, not the commit, and let restore-keys carry you across lockfile changes.

How do restore-keys and cache-hit actually behave?

Two details trip people up.

First, restore-keys are tried in order, and each is treated as a prefix. The first prefix that matches picks the most recently created cache among the matches. Put the most specific prefix first.

Second, the cache-hit output is 'true' only for an exact match on the primary key. A restore via restore-keys leaves it 'false' even though files were restored. If you gate npm ci on cache-hit != 'true' you will still run installs after a partial restore, which is usually what you want. If you gate it the other way, you will skip installs after a partial restore and ship a stale node_modules. Gating on cache-hit for something like node_modules is risky for exactly this reason; caching the package download directory (~/.npm) and always running npm ci is the safer shape.

Situation	Primary key	restore-keys	Result
Lockfile unchanged, same branch lineage	exact match	n/a	`cache-hit: true`, no save
Lockfile changed	miss	prefix match on older cache	`cache-hit: false`, files restored, new entry saved
First PR in a repo, main never cached	miss	no match	cold install, entry saved to PR ref only
Key includes `github.sha`	always miss	prefix match	works, but one new entry per run
Same key already exists	exact match	n/a	"Unable to reserve cache" warning on save

Takeaway: cache-hit means "exact key", not "something was restored", so don't use it to skip installs.

What about eviction and size limits?

As of late 2026, GitHub's documented defaults are: caches not accessed for 7 days are removed, and the repository total is capped (10 GB by default); when you exceed it, the oldest caches by last access are evicted first. Check the current numbers in the GitHub docs, since these have been adjustable. The practical effect: if you generate one big cache per PR, they can push out the default-branch cache that every PR depended on, and the next PR goes cold for no visible reason.

You can inspect what exists with the GitHub CLI:

gh cache list --limit 50
gh cache list --sort size_in_bytes --order desc

If the list is full of refs/pull/*/merge entries and the refs/heads/main entry is missing or old, you have found your problem.

To stop PRs from writing caches at all while still reading main's, split restore and save:

- uses: actions/cache/restore@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-

- run: npm ci

- uses: actions/cache/save@v4
  if: github.ref == 'refs/heads/main'
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}

Only main saves, so PR caches never crowd out the shared one. The tradeoff: a PR that changes dependencies pays the full install every time until it merges. For most teams that is the right trade.

Takeaway: one writer (the default branch) and many readers (PRs) is the cache topology that stays warm.

When is a built-in cache not enough?

The built-in cache is fine for package downloads. It is a poor fit for Docker layers (large, and the 10 GB quota fills fast) and for build outputs that are expensive to restore compared with rebuilding. For Docker, BuildKit's registry or GitHub Actions cache exporter is the usual route. If you would rather not manage cache quotas at all, a hosted runner service with persistent disks, such as Depot or Blacksmith, is the kind of product that takes this problem off your plate, at the cost of another vendor and bill to evaluate. I would only reach for that after measuring that cache restore time actually dominates your pipeline.

FAQ

Why does GitHub Actions say "Cache not found" when the cache exists?
Caches are scoped by branch. A PR can read its own ref, its base branch, and the default branch, but not caches saved by other branches or PRs.

Can I overwrite an existing GitHub Actions cache?
No. Caches are immutable per key. Change the key (for example by hashing your lockfile) or delete the old entry with gh cache delete before saving again.

Does cache-hit be true when restore-keys matched?
No. cache-hit is 'true' only when the primary key matched exactly; a restore-keys match restores files but reports 'false'.

Bottom line

If your cache never hits on PRs, check three things in order: does main run and save the cache, is your key derived from the lockfile rather than a static string or a commit SHA, and do your restore-keys prefixes match the primary key's prefix. For most small teams, a single saver on the default branch plus restore-only PRs is the simplest setup that stays fast. Reach for external cache services only once you have measured that restore time, not install logic, is the bottleneck.

Docker Compose depends_on Doesn't Wait for the Database: Fix "Connection Refused" on Startup

Libme — Thu, 29 Oct 2026 22:03:53 +0000

Short answer: plain depends_on only waits until the dependency's container has started, not until the process inside is ready. To make the app wait for Postgres, give the database a healthcheck and use depends_on with condition: service_healthy. Then add a connection retry in the app anyway, because a healthcheck narrows the race but cannot close it.

What does the failure look like?

You run docker compose up on a fresh checkout. The database container starts, the API container starts a moment later, and the API dies:

api-1  | psycopg2.OperationalError: connection to server at "db" (172.19.0.2), port 5432 failed: Connection refused
api-1  |     Is the server running on that host and accepting TCP/IP connections?
db-1   | PostgreSQL init process complete; ready for start up.

Run it again and it works, because the data volume now exists and Postgres boots in a second instead of ten. That "works on the second try" pattern is the signature of a startup race. Teammates on a clean clone hit it, and CI hits it every time.

The compose file that causes it usually looks like this:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD: example
  api:
    build: .
    depends_on:
      - db

Short-form depends_on controls start order. Compose starts db, sees the container is running, and starts api. It never asks whether Postgres is listening.

Takeaway: "container running" and "service ready" are different states, and short-form depends_on only knows the first.

How do you make Compose wait for Postgres?

Define a healthcheck on the database and make the dependency conditional on it:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: example
      POSTGRES_DB: app
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U app -d app"]
      interval: 3s
      timeout: 3s
      retries: 20
      start_period: 10s

  api:
    build: .
    depends_on:
      db:
        condition: service_healthy

Compose now holds api in "created" state until db reports healthy. If the database never becomes healthy within the retries, docker compose up exits with dependency failed to start: container db is unhealthy, which is a far better error than a stack trace from your app.

Takeaway: the long-form depends_on with condition: service_healthy turns "start order" into "readiness order".

Why does the healthcheck pass too early on a fresh volume?

This is the part most tutorials skip, and it bit me. The official postgres image, on the very first start with an empty data directory, runs its init scripts against a temporary server. As I understand the entrypoint script, that temporary server is configured to listen only on the Unix socket, not on TCP. When it finishes, it shuts down and the real server starts.

A healthcheck of just pg_isready -U app talks to the local Unix socket by default. During the init phase it can report "accepting connections" even though no TCP listener exists yet, and the server is about to restart. The result is a database that looks healthy, an app that connects over TCP to db:5432, and then either Connection refused or the database system is shutting down.

Adding -h 127.0.0.1 forces the check over TCP, which only succeeds once the final server is up. That one flag is why the example above uses it. If you use the socket form and only see failures on first boot (never on later boots with an existing volume), this is the cause.

Healthcheck command	Passes during init temp server?	Safe to gate the app on?
`pg_isready -U app`	Can, via Unix socket	No, flaky on first boot
`pg_isready -h 127.0.0.1 -U app -d app`	No, needs TCP listener	Yes
`psql -h 127.0.0.1 -U app -d app -c 'select 1'`	No, and proves auth works	Yes, slightly heavier

Takeaway: check readiness over the same transport your app will use, or the healthcheck can lie on first boot.

What about migrations that must finish before the app starts?

Waiting for the database is half of the problem. If a migration job must complete before the API serves traffic, Compose has a condition for that too:

services:
  migrate:
    build: .
    command: ["python", "-m", "alembic", "upgrade", "head"]
    depends_on:
      db:
        condition: service_healthy

  api:
    build: .
    depends_on:
      migrate:
        condition: service_completed_successfully
      db:
        condition: service_healthy

service_completed_successfully waits for the dependency to exit with code 0. If the migration fails, the API never starts. That is the behavior you want in a dev stack and in CI. (Alembic is just the example here; any one-shot command works.)

Takeaway: model one-shot setup work as its own service and gate on service_completed_successfully, instead of chaining shell commands in the app's entrypoint.

Do you still need retries in the app?

Yes. The healthcheck only governs the initial docker compose up. It does nothing when Postgres restarts later, when you run the app outside Compose, or in production on an orchestrator that has its own readiness semantics. An app that exits on its first failed connect is fragile everywhere except your laptop.

A minimal retry loop in Python:

import time
import psycopg2

def connect_with_retry(dsn: str, attempts: int = 10, base_delay: float = 0.5):
    for attempt in range(attempts):
        try:
            return psycopg2.connect(dsn, connect_timeout=3)
        except psycopg2.OperationalError:
            if attempt == attempts - 1:
                raise
            time.sleep(min(base_delay * 2 ** attempt, 5))

Keep the loop bounded and let the final error surface, so a genuinely broken config fails loudly instead of retrying forever. Pair it with restart: on-failure on the service if you want Compose to give the app another chance.

Which approach should you pick?

Approach	Handles first-boot race	Handles later DB restarts	Main drawback
Short `depends_on`	No	No	Only start order
`service_healthy` + healthcheck	Yes	No	Compose-only; healthcheck must be written correctly
`wait-for-it` style script in entrypoint	Mostly	No	Extra script to maintain; TCP-open is not the same as ready
App-level retry with backoff	Yes	Yes	Needs a code change in every service

If you live in Docker Compose for local development and CI, Docker Compose's service_healthy condition is the cleanest first fix because it needs no code changes. The durable answer is both layers: the healthcheck for a clean startup log, and retries in the app for everything the healthcheck cannot see. A drawback of leaning only on the healthcheck: it adds seconds to every cold start, and a too-tight retries value will mark a slow CI runner's database as unhealthy.

FAQ

Why does docker compose depends_on not wait for the database?
Short-form depends_on only waits for the dependency container to start, not for the process inside to be ready. Use condition: service_healthy with a healthcheck on the database.

What is the right pg_isready healthcheck for the Postgres Docker image?
Use pg_isready -h 127.0.0.1 -U <user> -d <db>. Forcing TCP avoids passing during the image's first-boot init phase, when only a temporary socket-only server is running.

Does depends_on condition service_healthy work with docker compose v2?
Yes. It is part of the Compose Specification supported by Docker Compose v2, as of late 2026, and it is not available in the legacy v1 file format with version: "2"/"3" semantics.

Bottom line

If your stack runs in Docker Compose, add a TCP-based pg_isready healthcheck and gate dependents on service_healthy, then put one-shot migrations in their own service with service_completed_successfully. Keep a bounded retry with backoff in the app so restarts and non-Compose environments behave too. Skip wait-for-it scripts unless you cannot use healthchecks, since they only prove a port is open.

DEV Community: Libme

Postgres Full-Text Search vs Meilisearch vs Typesense: Do You Need a Search Service for a Small App?

What does Postgres full-text search actually give you?

Where does Postgres search fall short?

How do Meilisearch and Typesense differ?

What is the hidden cost of adding a search service?

Which one should you pick?

FAQ

Bottom line

Related reading

GitHub Actions "Resource not accessible by integration": Fix It With the Smallest GITHUB_TOKEN Permission

What does the error actually look like?

Why did it work last month and fail now?

Which permission does each step need?

How do I scope it to one job?

Why does it still fail on fork pull requests?

What about reusable workflows and organization policy?

FAQ

Why does GitHub Actions say "Resource not accessible by integration"?

Is it safe to set workflow permissions to "read and write" for all workflows?

Why can't my workflow comment on pull requests from forks?

Bottom line

Related reading

Vercel vs Netlify vs Cloudflare Pages: Which Meter Will Your Frontend Hosting Bill Actually Grow On?

What is a frontend hosting bill actually made of?

How do the three differ on each meter?

When does SSR or image optimization become the real cost?

How do I estimate my bill before I commit?

Is it worth paying for the platform that matches my framework?

FAQ

Is Cloudflare Pages really free for unlimited bandwidth?

Why is my Vercel bill higher than the plan price?

Can I move a Next.js app off Vercel to avoid costs?

Bottom line

Related reading

LLM Returns Invalid JSON Once in a While: Structured Outputs vs Validate-and-Retry

What does the failure actually look like?

Step 1: Validate with a typed model instead of json.loads

Step 2: Retry with the error, not with the same prompt

When do provider-side structured outputs replace the loop?

Which approach should you use?

FAQ

Bottom line

Related reading

Docker Build Secrets: Why ARG and COPY Leak Tokens Into Image History, and the BuildKit Fix

How do I check whether my image already leaks a secret?

What does the BuildKit secret mount do differently?

Which method leaks what?

What about private git dependencies and CI?

What still goes wrong after you switch?

FAQ

Bottom line

Related reading

ENOSPC: System Limit for Number of File Watchers Reached — Raise the Limit, Shrink the Watch, or Poll?

What does ENOSPC mean when there's plenty of disk space?

How do I see who is using all the watches?

How do I raise the inotify limit permanently?

How do I make my tools watch fewer directories?

When should I switch to polling instead?

FAQ

Bottom line

Related reading

Docker Desktop Alternatives on Mac: OrbStack vs Colima vs Podman Desktop

Why look for an alternative at all?

How do the three compare?

How do you set up Colima without breaking existing scripts?

When is OrbStack worth it?

When does Podman Desktop make sense?

How should you decide?

FAQ

Is Colima a drop-in replacement for Docker Desktop?

Is Docker Desktop free?

Is OrbStack faster than Docker Desktop?

Bottom line

Related reading

pip "externally-managed-environment" Error: venv, pipx, or uv?

What does the error actually say, and why does it happen?

Which dead ends should you skip?

Which tool fits which job?

How do you set up a venv correctly (including the Debian gotcha)?

Step 1: Validate with a typed model instead of `json.loads`

When is `--break-system-packages` acceptable?

Is `pip install --break-system-packages` safe?

When is `--break-system-packages` acceptable?

Is `pip install --break-system-packages` safe?