Overpass API performance issues

aighes · 2026-04-25T23:49:24.665Z

NeatNit:

What kind of numbers could trigger 429? Probably this info is written somewhere and I missed it.

dev.overpass-api.de

Commons

Carnildo · 2026-04-26T03:38:01.259Z

NeatNit:

I was also going to suggest 401 Unauthorized: “semantically this response means ‘unauthenticated’. That is, the client must authenticate itself to get the requested response.”
It’s not a perfect fit since User Agent isn’t really authentication, but it’s closer to the true reason than 406 IMO.

“401 Unauthorized” is a bad choice: it’s supposed to trigger authentication in the user-agent, which for a web browser means popping up a username/password dialog.

tallelk78 · 2026-04-28T21:08:57.293Z

I’ve been running custom queries/python scripts in the QGIS console for the last year or so and I have been trying to troubleshoot this for over a week without I success.

I saw your post about “whitelisting” QGIS requests but unfortunately my scripts are still returning the 406. Can I still use “api.query(overpass_query)” or do I need send custom headers and change to a requests.post(url,data,headers)? I heavily manipulate the response data and would love to not modify those scripts as well.

*Full disclosure, I haven’t written a line of code (other than this) in over a decade so I’m definitely rusty!

tallelk78 · 2026-04-28T21:32:49.203Z

After Ionvia removed the QGIS UA from the block, did you notice a change? I can use the QuickOSM plugin but all of my python scripts are still failing.

Mateusz_Konieczny · 2026-04-29T04:48:27.101Z

tallelk78:

Can I still use “api.query(overpass_query)”

What documentation of this API in Qgis says about user agent? If nothing then reporting bug to whoever maintains it would be a good idea.
Though if problem is fixed in later release you will need to upgrade and if it is reported already - upvote or +1 it rather than creating a duplicate issue

ralley · 2026-04-30T05:00:07.662Z

Just wanted to add another note of thanks for enforcing the rules, overpass turbo is much more usable of late.

jidanni · 2026-05-02T01:40:02.188Z

Just curious, how can I still copy the link at the bottom of

Screenshot 2026-05-02 09.37.41481×684 21.5 KB

and use Wget on it, outside of Overpass Turbo?

If it’s a secret feel free to PM me.

I use about as much data as I have hair – no worry.

watmildon · 2026-05-02T04:46:39.728Z

As a consistent (if small time) creator of qa tools, visualizations, and maps that use Overpass, thank you thank you thank you! The de instance is alive and well again! Woooo!

Please let me know where to send thank you cards, beers, pizzas, etc!

Mateusz_Konieczny · 2026-05-02T07:01:23.492Z

wget should be able to set user agent, see it’s documentation

NeatNit · 2026-05-02T08:38:17.757Z

jidanni:

Just curious, how can I still copy the link at the bottom of

[snip]

and use Wget on it, outside of Overpass Turbo?

To set a custom User-Agent with wget, use --user-agent=AGENT (or the short version -U AGENT), for example:

wget -O - --user-agent=jidanni https://overpass-api.de/api/interpreter?data=...

But strangely enough, it seems that wget’s default user agent isn’t being blocked? I’m getting results without setting a custom user agent. I guess wget isn’t being used as much by bad actors?

For reference, based on wget -O - "https://httpbin.io/user-agent", my wget’s user agent is "Wget/1.21.4"

SomeoneElse · 2026-05-02T13:40:54.740Z

jidanni:

Just curious, how can I still copy the link at the bottom of [export on an overpass turbo screen] and use Wget on it, outside of Overpass Turbo?

I’ve just tried it here, and right-clicking the link in the browser, and pasting the results after wget -O=result.txt worked for me. The link includes the details of the actual Overpass server being used. The one I’m using doesn’t currently have performance problems, but (see rest of thread above) the ones with global coverage are being restricted in terms of what and who they will server. If you do get an error, you’ll want to investigate what it actually says.

lonvia · 2026-05-02T14:18:49.378Z

I believe that @jidanni is ~~complaining~~ observing that he can use Overpass with wget, not that he can’t.

That’s a correct observation. Neither curl nor wget are restricted at the moment. They do not seem to be the weapon of choice for the abusers. No promisses for the future, though. If you use them in scripts, always use the command line switch for adding a custom user agent. If you use them one-off manually, you’ll now know what to do if you suddenly get strange error messages.

jidanni · 2026-05-02T14:47:18.888Z

OK, it’s working again now, thanks.

DaveF · 2026-05-06T13:00:12.082Z

lonvia:

no improvement of the situation in sight in terms of users being a bit more mindful with their use of the Overpass API

I read a few threads on this subject and I confess I’m confused. Please clarify the exact problem. When did it manifest itself & have the restrictions which, I believe, have been in place for a while had any notable levels of success?

Mateusz_Konieczny · 2026-05-06T14:36:42.373Z

Exact problem is servers being overloaded, mostly by inept code.

Blocking bad traffic fixed problem, and intended use of overpass is now possible (as indicated by posts in this thread)

pnorman · 2026-05-09T06:00:08.539Z

DaveF:

Please clarify the exact problem.

The peak load on most or all public overpass servers is over their capacity. A significant portion of the load is from badly behaved automated queries.

DaveF:

When did it manifest itself

It’s been getting worse over time so there’s no one time you can point at and say it was good before then and bad after. Roland did a SOTM EU 2025 talk about it and that is when the issues became wider known.

DaveF:

have the restrictions which, I believe, have been in place for a while had any notable levels of success?

The anecdotal evidence here is that the restrictions placed about 20 days ago have helped.

gislars · 2026-05-09T12:41:41.560Z

It looks like https://www.osmhydrant.org/ is also affected, as I can see a 406 Not Acceptable response.

mvexel · 2026-05-14T17:43:18.131Z

With some delay and help from bradh (thanks) the Python overpass package now also requires setting a User-Agent.

Release 0.8.2 just went out.

https://codeberg.org/mvexel/overpass-api-python-wrapper/pulls/202
overpass · PyPI

Please update your Python project that uses overpass to use 0.8.2. Thanks.

Nakaner · 2026-05-15T12:48:40.521Z

gislars:

It looks like https://www.osmhydrant.org/ is also affected, as I can see a 406 Not Acceptable response.

Any application using Overpass as a backend is poorly designed if they always do almost the same queries and the queries are limited to simple things like points of interest (hydrants are POIs). Overpass is a good tool for one-time queries on OSM data. But it is the wrong tool for queries which are similar and differ by bounding box/spatial filter.

It’s the task of the maintainer of OSMHydrant to use their own backend like OpenCampingMap has been doing for many years. It has become much simpler with vector tiles. Just download the planet dump, filter it with osmium tags-filter, create vector tiles with tools like Tilemaker or Planetiler and serve the vector tiles. pyosmium-up-to-date can keep the planet up to date until the next vector tile set is to be generated. Filtering makes the planet shrink and reduce Tilemaker’s resource consumption to a smaller footprint.

IMHO, all those web applications which abuse Overpass as a backend for such simple queries should be blocked.

Kai_Johnson · 2026-05-15T14:50:10.303Z

The other option for applications like OSMHydrant that use Overpass as a back end is to use a local, private Overpass instance. Which has become much easier to do.