[07:47:33]
<selfhoster1312> is it a good idea for jinja rendering that we pass the whole environment directly?
[07:48:34]
<Aleks (he/him/il/lui)> ¯\_(ツ)_/¯
[07:49:14]
<selfhoster1312> i was thinking maybe we could at least have the manifest keys in the top-level too
[07:49:18]
<Aleks (he/him/il/lui)> ¯\_(ツ)_/¯ is it a good idea to run every app scripts as root?
[07:50:03]
<selfhoster1312> so at least { "manifest" => dict(...), "env" => dict(...) }
[07:50:38]
<selfhoster1312> but probably if it already exists at this point accessing the app settings (Python typed) would also be useful
[07:51:05]
<selfhoster1312> without having weird python/bash conversion methods such as `if val == "true"`
[07:52:25]
<selfhoster1312> my usecase is i want to do different templating based on app settings... for example for reverseproxy_ynh i could have different logic based on whether it's a unix socket or an IP:port, or whether the reverse proxy should inject trailing slash to all URLs, etc..
[07:54:23]
<selfhoster1312> it was a nightmare to do in bash with variables exported to the "plaintext" template, it's slightly less of a nightmare with the variables in the env that i can interpret from the jinja template, but it would be nice to use the actual values directly and enjoy the power of jinja :D
[07:55:12]
<selfhoster1312> PR welcome? or that sounds really weird to you?
[08:40:23]
<Aleks (he/him/il/lui)> I dont know what you want a PR about
[08:42:29]
<Aleks (he/him/il/lui)> There's already a --jinja option in ynh_add_config ?
[08:49:10]
<selfhoster1312> > Note that in YunoHost context, all variables are from shell variables and therefore are strings
that's what i want to fix in a PR :)
[08:49:37]
<selfhoster1312> https://github.com/YunoHost/yunohost/blob/eb14e404d6fdcb940f8e6048dc0a7c2799c4e75c/helpers/helpers.v1.d/templating#L406
> render(os.environ)
[08:50:09]
<selfhoster1312> i want to have `{ manifest => ..., env => ..., settings => ...}` in the templates not just os.environ (env)
[08:50:40]
<selfhoster1312> "everything is a string" is part of the bash nightmare and i wanna wake up ;)
[08:55:17]
<Aleks (he/him/il/lui)> I dont know how you're hoping to fix that .. We use bash and everything is a string, if you don't want everything to be a string we have to not use bash ? x_x
[08:55:31]
<selfhoster1312> excellent idea! :D
[08:55:45]
<selfhoster1312> joke aside, at least use real values in the templates because we have a real templating engine would be nice
[08:56:12]
<Aleks (he/him/il/lui)> > <selfhoster1312> i want to have `{ manifest => ..., env => ..., settings => ...}` in the templates not just os.environ (env)
I dont know what problem this address and this is going to be a breaking change for app already using the jinja option ? x_x
[08:56:34]
<selfhoster1312> who's using jinja? i'll make the PRs :)
[08:56:52]
<selfhoster1312> https://github.com/search?q=org%3AYunoHost-Apps%20jinja&type=code
[08:59:21]
<selfhoster1312> well the problem it addresses is begin to simplify the horrors we have because of bash usage... and therefore make it easier to use/document everything
[09:00:20]
<selfhoster1312> it's always confusing what an app setting value will be, depending if you're in yunohost python code, in settings.yaml, or in app install scripts... it's confusing what the name is due to bash rewrite rules, and it's confusing what the value is due to stringification
[09:00:23]
<Aleks (he/him/il/lui)> It's not just about PRs, because not every user upgrade all their apps as soon as we merge stuff
[09:00:56]
<selfhoster1312> i'd be happy to make helpers v2.1 just for this because it will really make packaging easier
[09:02:06]
<selfhoster1312> and it will make it somewhat easier to statically check templates before push... because you can verify that all keys used actually exist for example
[09:02:22]
<selfhoster1312> (something that's impossible if you do it from env which is populated dynamically)
[09:16:51]
<Aleks (he/him/il/lui)> still very fuzzy what you mean ... in the current packaging paradigm, many vars are populated dynamically yes, so it's not just about "using jinja" or "not using bash" to be able to magically validate templates
[09:19:00]
<Aleks (he/him/il/lui)> anyway imho the plan for templating is that we stick to the current templating format. I tried to argue something like 6 or 7 years ago that we should use a real template engine like jinja instead of a homemade format with less capabilities, there was nobody to back me up and we sticked to the `__FOO__` format. I'm not going to work on converting everything to jinja 6 years later, especially because in 99% of cases there's no gain in using jinja
[09:31:19]
<ljf> I was for jinja too in the past, but the team was divided, and we even called for a council decision to settle the matter... In the end, jinja was accepted (as the custom template engine), but was not integrated into the new helpers (ynh_add_conf) who manage the configs. And so today our custom template engine seems enough for our 99% needs.
We might deploy jinja by auto patching all apps packages (replace __VAR__ by {{ var }} is not so difficult), but for what gain? There are probably a few edge cases too.
Translated with DeepL.com (free version)
[09:41:19]
<selfhoster1312> ljf, you can't do conditional logic with __VAR__ like
```
{% if app_settings.foo == "whatever" %}
nginx_directive1 {{ app_settings.bar }}
{% elif app_settings.foo == "somethingelse" %}
nginx_directive2 {{ app_settings.bar }}
{% else %}
nginx_directive3 {{ app_settings.bar }}
{% endif %}
```
[09:44:26]
<Aleks (he/him/il/lui)> we are well aware that you can't do conditional block or for loops, the real question is how much do we really need this.... and how worth is it to rewrite every template in every app
[09:46:12]
<selfhoster1312> Aleks (he/him/il/lui), there's little injected variables in the environment... there's $app/$domain/$path (and even those i don't really see where they're documented?) and then there's the app settings... the former are just strings by nature so it's not much of a problem, but for app settings it would be helpful to have the actual value and not a creepy stringy version
[09:47:15]
<selfhoster1312> i mean most of the logic in form utils is due to dealing with bash weirdness, as i saw when rewriting it: https://github.com/selfhoster1312/yunohost-rs/blob/main/src/helpers/form.rs
[09:47:34]
<Aleks (he/him/il/lui)> settings are created as string values anyway, because they're created with `ynh_app_settings_set` most of the time ... i don't know where else you'd get a value that is not a creepy string ...
[09:49:25]
<selfhoster1312> yeah that's the thing... should it? or should we go to more declarative format via config panel?
[09:49:50]
<selfhoster1312> and maybe there's no need to rewrite *everything* if it goes in a new manifest/helpers version...
[09:52:41]
<Aleks (he/him/il/lui)> pretty sure there's an issue about this somewhere or it's in the packaging v3 wishlist
[09:53:13]
<selfhoster1312> there was jinja in the packaging v3 wishlist, but if we inherit all of bash's issues i don't see the point ^^
[09:53:22]
*selfhoster1312 digging further in the bugtracker
[09:54:21]
<Aleks (he/him/il/lui)> then let's plan "get rid of bash entirely" for packaging v4, that's totally gonna happen
[09:56:42]
<selfhoster1312> maybe not entirely, but 98% of bash usage in install scripts is redundant and a source of incorrectness, and would better be handled declaratively
[09:56:58]
<selfhoster1312> "Handle system confs declaratively" is already on the v3 wishlist :)
[09:57:59]
<selfhoster1312> anyway sorry for being annoying about rationalizing the foundations...
[09:58:08]
<Aleks (he/him/il/lui)> yes, reducing the amount of bash redundant stuff is one of the main goal of packaging v3, getting rid of the remove, backup/restore scripts which can be derived from declarative info, etc
[14:16:25]
<selfhoster1312> would it be a fair compromise to leave os.environ as global context for jinja templating, but add additional "reserved" keys like `app_settings` and `app_manifest` for example?
[14:17:25]
<selfhoster1312> (so not a breaking change)
[14:19:24]
<Aleks (he/him/il/lui)> i would still love to know what concrete use case you have in mind that require this ...
[14:19:45]
<selfhoster1312> conditional templating based on settings
[14:19:58]
<selfhoster1312> which i really struggled with for reverseproxy_ynh and why i left that app mostly dead
[14:20:50]
<Aleks (he/him/il/lui)> that's not a concrete use case, a concrete use case is "i'm working on app X in configuration Y and i want to use information Z"
[14:23:05]
<selfhoster1312> ok i'll make you an example :)
[14:23:10]
<selfhoster1312> (not right now though)