Bug 34239 - forums intermittently not available due to postgres errors, caused by AI scrapers.
Summary: forums intermittently not available due to postgres errors, caused by AI scra...
Status: RESOLVED FIXED
Alias: None
Product: Websites
Classification: Unclassified
Component: forums.mageia.org (show other bugs)
Version: trunk
Hardware: All Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Maat
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-05-02 15:10 CEST by Florian Hubold
Modified: 2025-05-25 16:52 CEST (History)
5 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments

Description Florian Hubold 2025-05-02 15:10:41 CEST
Description of problem:

Currently both the international and the german forums sometimes show only an error message instead of the forum:


General Error
SQL ERROR [ postgres ]

Errno 2: pg_connect(): Unable to connect to PostgreSQL server: FATAL: remaining connection slots are reserved for non-replication superuser connections at [ROOT]/includes/db/postgres.php line 108 []

An sql error occurred while fetching this page. Please contact an administrator if this problem persists.
Florian Hubold 2025-05-02 15:11:26 CEST

CC: (none) => doktor5000
Assignee: forums-bugs => sysadmin-bugs

Comment 1 Florian Hubold 2025-05-02 23:45:32 CEST
The database issue seems to also affect bugzilla, saw a similar issue several times today when checking e.g. https://bugs.mageia.org/buglist.cgi?chfield=%5BBug%20creation%5D&chfieldfrom=24h


> Software error:
> Can't connect to the database.
> Error: FATAL:  remaining connection slots are reserved for non-replication superuser connections
>   Is your database installed and up and running?
>   Do you have the correct username and password selected in localconfig?

> For help, please send mail to the webmaster (root@localhost), giving this error message and the time and date of the error.
Comment 2 sturmvogel 2025-05-10 10:16:11 CEST
Are there any investigations or mitigations ongoing? It takes ten minutes to make a comment for a bug as you can't access bugzilla, get disconnected whilst searching or making a comment. Is the database broken? Hardware underpowered? Any interest in making the situation better?
Comment 3 Dan Fandrich 2025-05-10 20:24:57 CEST
The working theory is that when the forum is being hit hard by bots the DB slows down and all the DB connections get used up (by more bots), eventually getting that error. It comes in waves and whenever I go to check it's all working fine. We've put some mitigations into place to deny bots on some high load web servers but not the forums. It's a hard problem since many of the bots don't play well and masquerade themselves as normal users and even piggyback on user IP addresses via surreptitious HTTP proxies. Do a search on something like "AI bots taking down my web server" and you can see that the problem is Internet-wide.

CC: (none) => dan

Comment 4 Marja Van Waes 2025-05-11 21:49:02 CEST
(In reply to Dan Fandrich from comment #3)
> We've put some mitigations into place to deny bots on some
> high load web servers but not the forums. It's a hard problem since many of
> the bots don't play well and masquerade themselves as normal users and even
> piggyback on user IP addresses via surreptitious HTTP proxies.

Maât can you do some mitigations on the forums VM?

CC: (none) => marja11, sysadmin-bugs
Assignee: sysadmin-bugs => maat-ml
Summary: forums intermittently not available due to postgres errors => forums intermittently not available due to postgres errors, caused by AI scrapers.

Comment 5 Florian Hubold 2025-05-11 23:44:44 CEST
(In reply to Dan Fandrich from comment #3)
> It comes in waves and whenever I go to check it's all working fine.

Thanks for the explanation, appreciated. Pascal was also working on that.

If you want to take a look, it's currently happening again.
Comment 6 Stephen Germany 2025-05-16 14:20:37 CEST
I couldn't get into the forums or bugzillla yesterday.

So far today it has been intermittent.

CC: (none) => stephengermany

Comment 7 Stephen Germany 2025-05-25 14:30:37 CEST
Seems to be back to normal now.
Comment 8 Dan Fandrich 2025-05-25 16:52:29 CEST
I've implemented some mitigations that seem to have worked: https://ml.mageia.org/l/arc/dev/2025-05/msg00082.html  I'm going to close this, so if things go south again, feel free to open a new bug.

Resolution: (none) => FIXED
Status: NEW => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.