[12:00:52]
<Yunohost Git/Infra notifications> [YunoHost documentation] [🔴 Down] Request failed with status code 500
[13:03:06]
<Aleks (he/him/il/lui)> https://mastodon.social/@cichy1173/109540235321931592 apparently this app is compatible with YunoHost somehow o.O https://play.google.com/store/apps/details?id=com.doyouhost.servercommander
[16:51:05]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[16:53:21]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[16:55:36]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[16:57:53]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:00:11]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:02:37]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:04:40]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:06:55]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:09:12]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:11:28]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:13:46]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:16:03]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:18:18]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:20:33]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:22:49]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:25:05]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:27:21]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:29:36]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:31:51]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:34:10]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:36:23]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:38:39]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:40:55]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:43:11]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:45:27]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:47:43]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:49:57]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:52:14]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:54:28]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:56:43]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[17:58:58]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:01:14]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:03:29]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:05:44]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:07:59]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:10:16]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:12:34]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:14:46]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:17:01]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:19:24]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:21:42]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:23:48]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:26:05]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:28:20]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:30:36]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:32:52]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:35:08]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:37:25]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:39:40]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:41:55]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:44:11]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:46:27]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:48:41]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:50:57]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:53:13]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:55:28]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[18:57:44]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:00:00]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:02:15]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:04:31]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:06:38]
<Maranda> 🤔
[19:06:47]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:09:02]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:11:18]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:13:33]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:15:49]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:18:07]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:20:26]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:22:38]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:24:53]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:27:09]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:29:25]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:31:40]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:33:55]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:36:12]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:38:29]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:40:45]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:43:00]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:45:17]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:47:32]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:49:55]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:52:03]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:54:18]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:56:46]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[19:58:49]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:01:06]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:03:21]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:05:36]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:07:52]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:10:08]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:12:24]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:14:39]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:16:54]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:19:10]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:21:24]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:23:42]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:26:00]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:28:15]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:30:29]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:32:45]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:32:51]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:34:59]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:37:18]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:38:52]
<Tag> ah
[20:39:06]
<Tag> lets try to reboot something then ?
[20:39:31]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:41:48]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:42:25]
<Tag> looks like postgres is doing some vacuum on dynette
[20:43:02]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:43:23]
<Tag> and there's a lot of dynette.cron.py running
[20:43:33]
<Aleks (he/him/il/lui)> ugh
[20:43:48]
<Tag> (ns1 is up so everything is fine atm)
[20:44:01]
<Tag> didn't reboot anything yet
[20:44:02]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:44:45]
<Aleks (he/him/il/lui)> holycow
[20:45:03]
<Tag> tempted to stop the vacuum, don't know how yet
[20:45:26]
<Aleks (he/him/il/lui)> sounds like the first cron is stuck on reload:
```
root 354832 354831 0 17:58 ? 00:00:00 | \_ /bin/sh -c /var/www/dynette/dynette.cron.py >> /var/log/dynette.log 2>&1
root 354833 354832 0 17:58 ? 00:00:02 | \_ /usr/bin/python /var/www/dynette/dynette.cron.py
root 354896 354833 0 17:59 ? 00:00:00 | \_ sh -c /usr/sbin/rndc reload
root 354897 354896 0 17:59 ? 00:00:00 | \_ /usr/sbin/rndc reload
```
[20:46:19]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:46:35]
<Aleks (he/him/il/lui)> and a shitload of `postgres: 9.6/main: dynette dynette [local] DELETE waiting` wtf :|
[20:47:06]
<Tag> try to not reboot challenge
[20:48:35]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:50:07]
<Tag> ```
2022-12-19 20:49:01.183 UTC [361188] dynette@dynette FATAL: sorry, too many clients already
2022-12-19 20:49:20.007 UTC [361203] dynette@dynette FATAL: sorry, too many clients already
2022-12-19 20:49:20.010 UTC [361204] dynette@dynette FATAL: sorry, too many clients alrea
```
`/var/log/postgresql/postgresql-9.6-main.log`
[20:50:30]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:50:51]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:51:21]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:51:39]
<Tag> starting at `2022-12-17 16:13:02.104 UTC [228174] dynette@dynette FATAL: sorry, too many clients already
`
[20:52:21]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:53:07]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:53:22]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:54:22]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:55:23]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:55:23]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:55:24]
<Tag> rah ouais ça log pas beaucoup la dynette
[20:56:28]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:57:23]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:57:38]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[20:58:10]
<Tag> Aleks (he/him/il/lui): can I restart the dynette lxc ?
[20:58:23]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:59:24]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[20:59:46]
<Tag> seems like postgres issue effectively
[20:59:54]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:00:26]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:01:24]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:02:09]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:02:24]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:03:26]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:04:27]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:04:27]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:05:25]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:06:27]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:06:40]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:07:26]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:08:26]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:08:57]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:09:44]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:10:26]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:11:13]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:11:37]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:12:29]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:13:27]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:13:27]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:14:28]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:15:27]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:15:43]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:16:31]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:17:28]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:18:00]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:18:28]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:19:29]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:20:20]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:20:29]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:21:29]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:22:30]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:22:31]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:23:31]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:24:31]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:24:47]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:25:31]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:26:31]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:27:02]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:27:33]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:28:44]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:29:18]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:29:34]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:30:33]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:31:33]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:31:33]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:32:33]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:33:34]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:33:48]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:34:34]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:35:34]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:36:04]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:36:34]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:37:35]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:38:21]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:38:35]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:39:35]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:40:36]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:40:36]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:41:36]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:42:36]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:43:21]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:43:37]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:44:37]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:45:08]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:45:37]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:46:38]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:47:25]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:47:39]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:48:38]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:49:38]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:49:41]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:50:39]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:51:39]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:51:56]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:52:40]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:53:40]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:54:11]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:54:41]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:55:40]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:56:27]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:56:41]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:57:41]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:58:41]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[21:58:42]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[21:59:42]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:00:42]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:00:58]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:01:42]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:02:42]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:03:14]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:03:43]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:04:44]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:05:29]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:05:44]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:06:44]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:07:44]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:07:45]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:08:45]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:09:45]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:09:59]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:10:45]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:11:46]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:12:15]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:12:46]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:13:46]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:14:30]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:14:46]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:15:47]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:16:46]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:16:47]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:17:47]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:18:48]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:19:01]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:19:48]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:20:48]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:21:17]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:21:49]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:22:49]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 500
[22:23:32]
<Yunohost Git/Infra notifications> [YunoHost NS0] [🔴 Down] queryA ETIMEOUT yoloswag.ynh.fr
[22:32:16]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[22:36:15]
<ljf[m]> should be fixed ?
[22:36:53]
<ljf[m]> i have clean log thin.500x.log file restarted bind9 (at the good moment) and restarted dyndns
[22:37:13]
<ljf[m]> (i mean `systemctl restart thin`)
[22:42:56]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[22:44:45]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[22:46:32]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[22:48:21]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[22:49:20]
<Aleks (he/him/il/lui)> Zbleuargh
[22:50:06]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] Request failed with status code 410
[22:52:58]
<ljf[m]> !#?$/!!!! /o\
[22:58:59]
<Tag> pouah CPU goes brrr
[22:59:35]
<Tag> load average: 5.11, 3.15, 2.72
[23:01:33]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:02:01]
<Tag> 80-100% CPU
[23:02:51]
<ljf[m]> ouai mais qu'est-ce qui explique ça
[23:03:09]
<Tag> thin server et named qui pompent à fond
[23:03:20]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:03:21]
<Tag> mais pourquoi... juste beaucoup de requêtes ?
[23:03:54]
<ljf[m]> j'avais pas l'impression
[23:05:09]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:06:57]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:08:45]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:09:57]
<Tag> ya encore 10 cron lancés en même temps, je sais pas si c'est très normal
[23:10:27]
<Tag> et on reçoit pas tant de requêtes que ça sur bind
[23:10:33]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:10:41]
<Tag> là c'est surtout dynette et postgres qui pompent tout
[23:10:55]
<Tag> bind a l'air de bien tenir maintenant somehow
[23:12:21]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:13:08]
<ljf[m]> je viens de capter, en fait c'est pas la dynette, les ressources sont partagées
[23:13:31]
<ljf[m]> arf non j'ai rien dit
[23:14:10]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:15:58]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:16:02]
<Tag> la dynette n'a pas l'air de log grand chose là c'est chelou
[23:16:30]
<Tag> et pourtant thin qui tourne à fond
[23:16:35]
<Tag> pas normal normal ça
[23:16:54]
<ljf[m]> +1
[23:17:32]
<Tag> d'où peuvent venir ces requêtes ?
`::1 - - [19/Dec/2022:23:16:54 +0000] "GET /all/nohost.me HTTP/1.0" 200 5326529 30.5850`
[23:17:46]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:19:03]
<ljf[m]> je restart thin, pour pouvoir voir ce qu'il y a derriere cette requête
[23:19:34]
<Yunohost Git/Infra notifications> [YunoHost DynDNS] [🔴 Down] timeout of 48000ms exceeded
[23:20:08]
<Tag> c'est toi la 403 j'imagine
[23:20:14]
<ljf[m]> bizarre si ça fait une 200, ça doit venir d'une ip autorisée
[23:20:30]
<ljf[m]> donc d'un trucs en interne dans l'infra
[23:20:33]
<Tag> bon là ça repart
[23:20:52]
<Tag> sans trop de secousses pour l'instant
[23:21:15]
<Tag> je note pour la prochaine fois : reboot jusqu'à ce que ça doit stable
[23:21:19]
<Tag> 😁
[23:21:49]
<ljf[m]> https://github.com/YunoHost/dynette/blob/master/dynette.cron.py#L115
[23:22:07]
<ljf[m]> la requête vient de la tache cron
[23:22:53]
<ljf[m]> moui peut être que ça marche mais si c'est pour 5minutes...
[23:23:22]
<ljf[m]> (bon perso je vais quand même pas y passer la soirée car j'ai une mise en prod ce soir
[23:23:58]
<ljf[m]> (enfin cette nuit du coup, à cause de la réu FFDN et puis ça...)
[23:24:17]
<Tag> je surveille encore un peu, merci pour le coup de main ce soir déjà 👍️
[23:40:12]
<Tag> ```
Traceback (most recent call last):
File "/var/www/dynette/dynette.cron.py", line 116, in <module>
result = json.loads(str(urlopen(url +'/all/'+ domain).read()))
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
```
in `/var/log/dynette.log` (the cron)
[23:41:05]
<Tag> ah not relevant