[01:29:10]
<m606> ah yes it respawns - at least i can backup the doc! But so we are not supposed to have it anymore, right?
[01:30:13]
<m606> I don't get this part ? is .github a file extension?
[05:59:16]
<tituspijean[m]> > <@m606:matrix.org> I don't get this part ? is .github a file extension?
He is referring to https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file and https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/creating-a-pull-request-template-for-your-repository
[08:29:02]
<Yunohost Git/Infra notifications> [penpot_ynh] orhtej2 merged [pull request #168](https://github.com/YunoHost-Apps/penpot_ynh/pull/168): Upgrade sources
[08:29:04]
<Yunohost Git/Infra notifications> [penpot_ynh] orhtej2 deleted branch ci-auto-update-sources-251126
[10:10:28]
<anubister> hello, je pensais avoir les droits pour accepter les PR mais en fait non, et je ne sais pas qui a les droits sur cette app, serait-il possible d'accepter la PR (je l'ai testée) : https://github.com/YunoHost-Apps/movim_ynh/pull/95
[10:17:09]
<kayou> done
[10:19:12]
<kayou> https://ci-apps-dev.yunohost.org/ci/job/15084
[10:19:33]
<kayou> je vais attendre la CI
[10:30:15]
<kayou> merged in master
[11:06:16]
<Yunohost Git/Infra notifications> [synapse_ynh] Josue-T pushed to update: Upgrade synapse to 1.143.0 ([5828d67b](https://github.com/YunoHost-Apps/synapse_ynh/commit/5828d67bdb0071535ca2b551f98c8de411c6a242))
[11:07:58]
<Yunohost Git/Infra notifications> [synapse_ynh] Josue-T opened [pull request #592](https://github.com/YunoHost-Apps/synapse_ynh/pull/592): Update
[11:15:24]
<anubister> merci !
[11:15:40]
<Yunohost Git/Infra notifications> App moncycle rises from level 0 to 8 in job [#25118](https://ci-apps.yunohost.org/ci/job/25118) !
[11:59:33]
<Aleks (he/him/il/lui)> ah thank god 😅
i mean hmmmm i guess you can keep using it and next time i iterate on the topic of disabling unecessary github repo features i should be more careful ... though it still sounds a bit meh to me to put this in a github wiki like ... i'm not sure how people are supposed to guess that they should go read the wiki for those specific repo
if somebody has time / energy / motivation i would encourage to improving the autoupdate mechanism to be able to declare something like a `autoupdater.note` that would be added to the PR created by the bot maybe
[13:27:09]
<m606> So far in addition to the [wiki](https://github.com/YunoHost-Apps/it-tools_ynh/wiki/How-to-apply-upstream-upgrades-to-this-package) (which is giving a broader perspective), I had also set the mechanism you mention to add a short checklist to the PR but [using Github Actions](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/.github/workflows/ynh-build-on-upstream-update.yml#L76-L83). However it happens to not be always very reliable - for instance this Github Action did no trigger for the [last yuno-bot upgrade PR here](https://github.com/YunoHost-Apps/it-tools_ynh/pull/73) due to the new PR event [not being even detected](https://github.com/YunoHost-Apps/it-tools_ynh/actions/workflows/ynh-build-on-upstream-update.yml) as it should... I don't get why. The yuno-bot PR was merged ~2h after its release, maybe the "Github cron" for detecting events has a larger period between each run?
That shouldn't be too much for me to work on that autoupdater.note which would be apparently more reliable than the Github Actions. However would you have other usecase in mind than [this one](https://github.com/YunoHost/issues/issues/2281#issuecomment-2928087687)? If not, I wonder whether I should start (slowly) working on a better integration of the prebuild mechanism. But there are still some design questions to me following the specs ideas you wrote [there](https://github.com/YunoHost/issues/issues/2281#issuecomment-2931277536).
[13:28:17]
<m606> So far in addition to the [wiki](https://github.com/YunoHost-Apps/it-tools_ynh/wiki/How-to-apply-upstream-upgrades-to-this-package) (which is giving a broader perspective), I had also set the mechanism you mention to add a short checklist to the PR but [using Github Actions](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/.github/workflows/ynh-build-on-upstream-update.yml#L76-L83). However it happens to not be always very reliable - for instance this Github Action did no trigger for the [last yuno-bot upgrade PR here](https://github.com/YunoHost-Apps/it-tools_ynh/pull/73) due to the new PR event [not being even detected](https://github.com/YunoHost-Apps/it-tools_ynh/actions/workflows/ynh-build-on-upstream-update.yml) as it should... I don't get why. The yuno-bot PR was merged ~2h after its release, maybe the "Github cron" for detecting events has a larger period between each run?
That shouldn't be too much for me to work on that autoupdater.note which would be apparently more reliable than the Github Actions. However would you have other usecase in mind than [this one](https://github.com/YunoHost/issues/issues/2281#issuecomment-2928087687)? If not, I wonder whether I should rather start (slowly) working on a better integration of the prebuild mechanism. But there are still some design questions to me following the specs ideas you wrote [there](https://github.com/YunoHost/issues/issues/2281#issuecomment-2931277536).
[13:29:06]
<m606> So far in addition to the [wiki](https://github.com/YunoHost-Apps/it-tools_ynh/wiki/How-to-apply-upstream-upgrades-to-this-package) (which is giving a broader perspective), I had also set the mechanism you mention to add a short checklist to the PR but [using Github Actions](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/.github/workflows/ynh-build-on-upstream-update.yml#L76-L83). However it happens to not be always very reliable - for instance this Github Action did no trigger for the [last yuno-bot upgrade PR here](https://github.com/YunoHost-Apps/it-tools_ynh/pull/73) due to the new PR event [not being even detected](https://github.com/YunoHost-Apps/it-tools_ynh/actions/workflows/ynh-build-on-upstream-update.yml) as it should... I don't get why. The yuno-bot PR was merged ~2h after its release, maybe the "Github cron" for detecting events has a larger period between each run?
That shouldn't be too much for me to work on that autoupdater.note which would be apparently more reliable than the Github Actions. However would you have other usecase in mind than [this one](https://github.com/YunoHost/issues/issues/2281#issuecomment-2928087687)? If not, I wonder whether I should rather start (slowly) working on a better integration of the prebuild mechanism. But there are still some design questions to me following the specs ideas you summarized [there](https://github.com/YunoHost/issues/issues/2281#issuecomment-2931277536).
[13:37:04]
<Aleks (he/him/il/lui)> eerhg yeah the "prebuild" topic is much broader than just tweaking the PR message, there are plenty of design questions, it's somewhat related to packaging v3 etc...
[13:37:41]
<m606> https://upptime.js.org/blog/2021/01/22/github-actions-schedule-not-working/
[13:37:58]
<Aleks (he/him/il/lui)> (or we could somehow design something independent from packaging v3 but in my mind on the long term, the "build" logic should be the same wether build happens locally or remotely/pre-building)
[13:38:55]
<m606> I concur... better doing the work with Packaging v3 in mind
[14:10:36]
<m606> But so, for now, to link to documentation of the [full process](https://github.com/YunoHost-Apps/it-tools_ynh/wiki/How-to-apply-upstream-upgrades-to-this-package) (that would be too much to add in a PR) should I use wiki or .github?
[14:11:39]
<m606> I can also link to the doc in [that github repo](https://github.com/oleole39/prebuild-app), but that's outside of Yunohost-Apps org...
[14:34:49]
<m606> BTW in the current design of prebuild mechanism using Github Actions you already have a build script that can more or less be run indifferently either locally and on Github cloud. What is lacking currently for it to be fully integrated in the YNH install workflow is the step in this workflow where you could choose between both options (and probably adding a default behavior which should to my mind rely on a global setting to be set in YNH config - always local build, always prebuild or ask everytime), i.e.
1. if `ynh_build` resource exists in the manifest:
- check for global default setting and according to it trigger local build script or use prebuild or ask
- trigger build script, directly install using prebuild resource, or ask what to do
2. else trigger build script
[14:37:46]
<m606> (this script for instance: https://github.com/YunoHost-Apps/it-tools_ynh/blob/master/scripts/build)
[14:43:26]
<Aleks (he/him/il/lui)> hmmmmmm but i think the issue is much broader than just this ...
- ideally we don't want to depend on Github infrastructure and afaik the way Github actions are used right now they'll only build stuff for x64 arch ?
- and we probably don't want to have the build logic being "hidden" in the .github folder rather than the scripts/ folder
- right now there's only a handful of apps that we prebuild ourselves, but in general the goal would be that any app with an "npm install / npm build" that takes a significant amount of time / resource should be built externally .. or maybe even all of them because "npm install" is so fucking slow and dependencies might move over time or idk ...
- there's a relation with the backup/restore flow where ideally we don't want to backup the node_modules folder (i dont know, to be discussed) but if we don't backup it, we need some guarantee that we can redownload the dependencies etc .. or maybe we should use a proper cache system locally like pnpm idk
- anyway, we'll probably want to build stuff for several apps, and for all the relevant architecture, locally on our infrastructure, which means we need to design a system where we can run the build procedures in a safe environment where malicious code would not compromise the host
- and we need a way to make the builds available, some sort of CDN, we need to define how we index them, how reproducible they are, and we need to define a policy of how long we'll keep builds etc
- to some extend there's also the thing of "maybe we should backup the .tar.gz and other sources from upstream because we regularly see cases where they disappear and that makes the app spontaneously break"
[14:43:35]
<Aleks (he/him/il/lui)> (but that's also me being jusqu'auboutiste)
[15:22:20]
<m606> Matrix doesn't make it easy to have such point by point conversations and to archive them, but shortly, in the same order:
- IDK, but so far i've prebuilt only NPM webapps for which arch was not a topic.
- Yes when I first "released" this logic this year, you and others said you'd rather see it as a python script being part of YNH. I agree and could work on this, but it comes back to the question on how should I work on this now given your ongoing work on Packaging v3 ?
- Yes, that's my goal too, there are hundreds of npm apps in the catalog (not to mention Go apps and potentially others) which are simply not available to hosters with little RAM/storage available due to the build requirements. As for the backup topic, global caching could be an improvement, but waiting for this and to keep it simple and light in relation to storage (although it would then ask more from network), I would rather remove the local build folder's node_modules folder straight after build from whithin the build script.
- Yes, however remote build on YNH infra sounds like a more long-term topic. As a middle step, we could have YNH-integrated remote build logic (cf. point 2) together with Github Actions to use Github resources for build.
- Yes, but those questions are mostly also for remote build on YNH infra (cf. my answer to point 4). Using Github infra, I kind of believe to have seen somewhere reproducible builds could be made there but that's vague and I don't know much more for now. Even if that was not possible, that would be a downside of the Github source, and careful hosters could still use only local build if they prefer so.
[15:22:42]
<m606> Matrix doesn't make it easy to have such point by point conversations and to archive them, but shortly, in the same order:
- IDK, but so far i've prebuilt only NPM webapps for which arch was not a topic.
- Yes when I first "released" this logic this year, you and others said you'd rather see it as a python script being part of YNH. I agree and could work on this, but it comes back to the question on how should I work on this now given your ongoing work on Packaging v3 ?
- Yes, that's my goal too, there are hundreds of npm apps in the catalog (not to mention Go apps and potentially others) which are simply not available to hosters with little RAM/storage available due to the build requirements. As for the backup topic, global caching could be an improvement, but waiting for this and to keep it simple and light in relation to storage (although it would then ask more from network), I would rather remove the local build folder's node_modules folder straight after build from whithin the build script.
- Yes, however remote build on YNH infra sounds like a more long-term topic. As a middle step, we could have YNH-integrated remote build logic (cf. point 2) together with Github Actions to use Github resources for build.
- Yes, but those questions are mostly also for remote build on YNH infra (cf. point 4). Using Github infra, I kind of believe to have seen somewhere reproducible builds could be made there but that's vague and I don't know much more for now. Even if that was not possible, that would be a downside of the Github source, and careful hosters could still use only local build if they prefer so.
[15:26:47]
<Aleks (he/him/il/lui)> yeah it's a complex discussion, my main point was that imho it's not possible to design something that will still be relevant in 1-2 years from now considering that packaging v3 is supposed to play a role in this, i don't think it's worth investing energy in the current ad-hoc, per-app github actions..
[15:40:14]
<m606> you mean even considering point 2 in my last message, i.e. translating [this logic](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L44-L357) in Python to add it to YNH ? That this logic wouldn't be of any use later on? Because the per-app manual configuration is only [this bit](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L14-L43) (i.e. not a lot to me, even though of course it adds some complexity at scale) and only this drives the Github actions files which are meant to be the same for all apps. I wonder whether this could maybe pave the way to an improved V3 mechanism since most of the per-app config will I believe be required anyway in such V3 mechanism (vars such as node version and target upstream branch name or optional customizations commands).
[15:46:14]
<Aleks (he/him/il/lui)> uuuuuuuuuuh idk i'm lost into what the script does exactly though it's pretty cool that the script is easily parametrizable but urrgh it "only" seems to apply to nodejs but python venvs are supposedly an important use case too uuuuuh idk i don't have the mental space right now to dig right now and we'd need to design / speak about this out loud if we really want to move toward something idk
[15:47:20]
<Aleks (he/him/il/lui)> ah i see i didnt realize there was the "build instructions" function defining the actual build procedure
[15:51:19]
<Aleks (he/him/il/lui)> (i'm mostly confused about uuuuuh how this would get into yunohost core if the point of this code is to be ran by maintainers or github CI to build and upload the artifacts to some github release ? sorry for all the confusion x_x)
[16:08:28]
<m606> It is slightly complex. Let me know if this is not the good moment for you to discuss this.
Yes the bash script so far only addresses nodejs (which nevertheless already account for ~400 apps or so and could be deployed for those apps only at first, i.e. not offering option for prebuild files for other ). However it was made to be easily adapted to other cases later on (say Go and Python venvs). There is this [parameter `runtime_environement`](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L23) which so far only accepts `nodejs` or `none`, but which could be added support for `go`and `pyvenv` for instance. For this it would mostly take to add the correponding case in [this section of the script](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L168-L182) and do some testing.
So the script doesn't cover all potential use cases for now, but is thought to be improved easily in this regard, and could be deployed to cover only one case (nodejs) and gradually cover more of them (go, python venvs, etc.)
[16:18:47]
<m606> Then how it works it the following, based on the [per-app input](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L14-L42), it manages the whole build process "locally" and outputs a tar.gz file which is the built asset. Now "locally" refers to the location which triggered the script. If you run the script with `./myapp_ynh/scripts/build` on your local machine then that what would be what is commonly referred to as locally (option 1 in YNH - local build). If however the script is run by a github worklow, then "locally" refers to the Github server (option 2 in YNH - prebuild file using Github cloud).
[16:20:43]
<m606> the github actions workflow added to .github folder in each app repo acts as mere generic orchestrator for this build script (the build script passes the specific variables to the github workflow to create PR and all).
[16:22:03]
<m606> so what is lacking for now is the choice management (local or prebuild) script and orchestrator (to simply trigger ./myapp_ynh/scripts/build if local build option is chosen) within YNH core
[16:22:55]
<m606> let me know if this still confuses you
[16:23:33]
<m606> Then how it works it the following - based on the [per-app input](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L14-L42), it manages the whole build process "locally" and outputs a tar.gz file which is the built asset. Now "locally" refers to the location which triggered the script. If you run the script with `./myapp_ynh/scripts/build` on your local machine then that what would be what is commonly referred to as locally (option 1 in YNH - local build). If however the script is run by a github worklow, then "locally" refers to the Github server (option 2 in YNH - prebuild file using Github cloud).
[16:24:40]
<m606> Then how it works it the following - based on the [per-app input](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L14-L42), it manages the whole build process "locally" and outputs a tar.gz file which is the built asset. Now "locally" refers to the location which triggered the script. If you run the script with `./myapp_ynh/scripts/build` on your local machine then it refers to what would commonly be referred to as locally (option 1 in YNH - local build). If however the script is run by a Github worklow, then "locally" refers to the Github server (option 2 in YNH - prebuild file using Github cloud).
[16:45:10]
<Aleks (he/him/il/lui)> yeah i don't know what to say, there's a lot of overlap woth packaging v3 in which the new `build()` function that'll live in `scripts.sh` is supposed to be able to be ran locally or in an externalized build too, with modalities still to be defined (eg the `build()` function will probably need to fetch the source with `ynh_setup_source` which implies the yunohost helpers and resources (e.g. proper node/npm version) are available during the externalized build context etc ... or switching between local build vs prebuilt ... and also clearly we don't want to use GitHub as a CDN more than the current temporary-ish situation we have right now with the app that really need pre-building
[16:46:24]
<Aleks (he/him/il/lui)> there was also discussion to explore using the Nix formalism as build mechanism because it already solve the issue of reproducibility and can create AppImage which may be useful, but it's still pretty vague and we need to check how complex and relevant that would be etc
[17:01:10]
<Yunohost Git/Infra notifications> [synapse_ynh] Josue-T pushed to testing: Upgrade synapse to 1.143.0 ([5828d67b](https://github.com/YunoHost-Apps/synapse_ynh/commit/5828d67bdb0071535ca2b551f98c8de411c6a242))
[17:01:11]
<Yunohost Git/Infra notifications> [synapse_ynh] Josue-T merged [pull request #592](https://github.com/YunoHost-Apps/synapse_ynh/pull/592): Update
[17:01:12]
<Yunohost Git/Infra notifications> [synapse_ynh] Josue-T pushed to testing: Auto-update READMEs ([512abbbb](https://github.com/YunoHost-Apps/synapse_ynh/commit/512abbbb6df22f7c88667a696892706b450d3705))
[17:01:26]
<Yunohost Git/Infra notifications> [synapse_ynh] Josue-T deleted branch update
[19:27:11]
<m606> OK well I guess I can't do much more on this for now then
[21:48:58]
<m606> Then how it works is the following - based on the [per-app input](https://github.com/YunoHost-Apps/it-tools_ynh/blob/26ddd0079442d53fbf30ff1ed4e9fb4eae7dcffd/scripts/build#L14-L42), it manages the whole build process "locally" and outputs a tar.gz file which is the built asset. Now "locally" refers to the location which triggered the script. If you run the script with `./myapp_ynh/scripts/build` on your local machine then it refers to what would commonly be referred to as locally (option 1 in YNH - local build). If however the script is run by a Github worklow, then "locally" refers to the Github server (option 2 in YNH - prebuild file using Github cloud).
[23:23:17]
<Yunohost Git/Infra notifications> [stirling-pdf_ynh] yunohost-bot opened [pull request #94](https://github.com/YunoHost-Apps/stirling-pdf_ynh/pull/94): Upgrade to v2.0.1
[23:27:23]
<Yunohost Git/Infra notifications> Autoupdater just ran, here are the results:
- 40 pending update PRs
- 13 new apps PRs: cloudlog, discourse, filerise, grist, invoiceninja5, jackett, jeedom, localai, matomo, nodebb, readeck, stirling-pdf, teampass
- 12 failed apps updates: calcom, calibreweb-automated, element-call, fluffychat, fusion, lidarr, litechat, loops, pollaris, radarr, tube, yarr
See the full log here: https://paste.yunohost.org/raw/zewimenova
Autoupdate dashboard: https://apps.yunohost.org/dash?filter=autoupdate