From dbe477b2052c248eb0c0b27e9366e91eb774fe29 Mon Sep 17 00:00:00 2001
From: Clark Boylan <clark.boylan@gmail.com>
Date: Thu, 1 Feb 2024 13:34:01 -0800
Subject: [PATCH] Increase gitea db connection limit

By default our mariadb database for gitea nodes limits itself to a
maximum of 100 connections. We've seen errors like this:

 ...eb/routing/logger.go:102:func1() [I] router: completed POST /openstack/requirements/git-upload-pack for 127.0.0.1:50562, 500 Internal Server Error in 2.6ms @ context/user.go:17(web.gitHTTPRouters.UserAssignmentWeb)
 ...ules/context/repo.go:467:RepoAssignment() [E] GetUserByName: Error 1040: Too many connections

And after reading gitea's source code this appears to be related to user
lookups to determine if the user making a request against a repo owns
the repo. To do this gitea does a db request to lookup the user from the
request and when this hits the connection limit it bubbles up the mysql
error 1040: Too may connections error.

This problem seems infrequent so we double the limit to 200 which is
both much larger but still a reasonable number.

We also modify the test that checks for gitea server errors without an
http 500 return code to avoid it matching this change improperly. This
was happening because the commit message ends up in the rendered pages
for system-config in the test gitea.

Change-Id: If8c72ab277e88ae09a44a64a1571f94e43df23f8
---
 playbooks/roles/gitea/files/99-max_conn_my.cnf         | 2 ++
 playbooks/roles/gitea/tasks/main.yaml                  | 7 +++++++
 playbooks/roles/gitea/templates/docker-compose.yaml.j2 | 1 +
 testinfra/test_gitea.py                                | 2 +-
 4 files changed, 11 insertions(+), 1 deletion(-)
 create mode 100644 playbooks/roles/gitea/files/99-max_conn_my.cnf

diff --git a/playbooks/roles/gitea/files/99-max_conn_my.cnf b/playbooks/roles/gitea/files/99-max_conn_my.cnf
new file mode 100644
index 0000000000..6e4910da5a
--- /dev/null
+++ b/playbooks/roles/gitea/files/99-max_conn_my.cnf
@@ -0,0 +1,2 @@
+[mysqld]
+max_connections=200
diff --git a/playbooks/roles/gitea/tasks/main.yaml b/playbooks/roles/gitea/tasks/main.yaml
index 666e275514..3965bccaab 100644
--- a/playbooks/roles/gitea/tasks/main.yaml
+++ b/playbooks/roles/gitea/tasks/main.yaml
@@ -24,6 +24,13 @@
   template:
     src: app.ini.j2
     dest: /var/gitea/conf/app.ini
+- name: Write mariadb conn limit config file
+  copy:
+    src: 99-max_conn_my.cnf
+    dest: /var/gitea/conf/99-max_conn_my.cnf
+    owner: root
+    group: root
+    mode: 0644
 - name: Install distro packages
   package:
     name:
diff --git a/playbooks/roles/gitea/templates/docker-compose.yaml.j2 b/playbooks/roles/gitea/templates/docker-compose.yaml.j2
index 542e70372e..504a79bb1d 100644
--- a/playbooks/roles/gitea/templates/docker-compose.yaml.j2
+++ b/playbooks/roles/gitea/templates/docker-compose.yaml.j2
@@ -14,6 +14,7 @@ services:
       MYSQL_PASSWORD: "{{ gitea_db_password }}"
     volumes:
       - /var/gitea/db:/var/lib/mysql
+      - /var/gitea/conf/99-max_conn_my.cnf:/etc/mysql/conf.d/99-max_conn_my.cnf:ro
     logging:
       driver: syslog
       options:
diff --git a/testinfra/test_gitea.py b/testinfra/test_gitea.py
index a3eb86be2c..260d681ed1 100644
--- a/testinfra/test_gitea.py
+++ b/testinfra/test_gitea.py
@@ -130,7 +130,7 @@ def test_no_500_template_content(host):
                        '--resolve gitea99.opendev.org:3081:127.0.0.1 '
                        'https://gitea99.opendev.org:3081' + path)
         assert 'status-page-500' not in cmd.stdout
-        assert 'Internal Server Error' not in cmd.stdout
+        assert '<title>Internal Server Error' not in cmd.stdout
 
 def test_gitea_screenshots(host):