RFC: an improved ki18n_install
Hey all,
for context please see [1] and [2].
[1]: https://bugs.kde.org/show_bug.cgi?id=460245
[2]: https://discourse.cmake.org/t/feature-request-add-custom-command-all/7609
The gist is that me and Igor are annoyed by `ki18n_install` abusing
`add_custom_target`, which makes the build always dirty. I.e. every time we
run `ninja` we will, at the very least, run the two mo & ts generation
targets. For kdevelop, these do non-trivial amounts of work that takes time
even on a beefy laptop:
```
$ ninja && time ninja
[2/2] Generating mo...
[2/2] Generating mo...
real0m1,780s
user0m5,742s
sys 0m2,327s
```
I would like to fix this, but first want to get feedback from the KDE
developers at large.
First of all: Are all projects using ki18n_install in the way we do? Why is
no-one else complaining about this so far? Are your po files so small that
this doesn't take a significant amount of time? Or are there potentially just
so few of them in your project? KDevelop is heavily plugin based which means
we have tons of po files. 2464 *.po files to be exact...
Secondly: does anyone know how to best approach a solution to this issue? The
problem is that I don't see an easy way to solve it: While Kyle Edwards was
nice enough to show me a way to tell CMake to not make the custom target
always-dirty, `k18n_install` suffers from another design deficiency: It uses
globbing internally and has an undefined amount of inputs and outputs, which
basically makes it impossible for us to leverage `add_custom_command(OUTPUT`
here, or?
As such, it seems like one would need a different approach to this problem
anyways. Globbing is a known-evil in cmake land, and I would personally prefer
to have the inputs explicitly listed, just like we do for source files etc.
Because we have many languages though, listing all possible *.po or *.ts files
is cumbersome. Instead, what about we maintain the list of known languages in
the ki18n framework? Then we could have something like
```
ki18n_target(
# the target for which we are handling messages
TARGET KF5I18n
# the translation domain, added as compile_options and to find files
TRANSLATION_DOMAIN ki18n5
# the root po folder location, to look for the .po/.js files
PO_FOLDER ${KI18n_SOURCE_DIR}/po
)
```
Internally, this would do what is currently done manually:
```
target_compile_options(KF5I18n PRIVATE -DTRANSLATION_DOMAIN=\"ki18n5\")
```
Then it would iterate over the list of known languages and try to find the .po
or .js file. For every match, we would add_custom_target to generate the
corresponding .mo/.ts file and add the reverse dependency.
What do others say to this approach?
Thanks
--
Milian Wolff
http://milianw.de
Re: MR Gardening - A discussion, please leave your input!
Hi, for GCompris, we don't have a lot of MR and even if some are old, we still plan to take over them at some point (we know which ones are unmaintained) so I think it's preferable to not add messages. In a general manner, as said by Carl, MR should not be closed automatically. Cheers, Johnny Le dim. 5 mars 2023 à 11:14, AnnoyingRains a écrit : > For a short amount of time now, there have been some small-scale > trials of replying to old MRs with a reminder, and suggesting that the > author closes the MR if it is either no longer needed or if it needs > more work and the author does not have time for it. > > This has appeared to have a positive impact on the state of KDE > software from this (albeit limited) trial. Some MRs have had renewed > interest, others have admitted that they had forgotten that the MR > even existed. > > We did consider closing MRs if there was no activity after our ping > message. **We are no longer planning on doing this**, as it is more > destructive than helpful. All decisions on if a MR should be closed > will be left with the maintainers and the person who opened the MR. > > So, we need a proper discussion about this, should we send these > reminder messages at all? If so, how old should an MR be before > sending this reminder? Should closing the MR even be suggested in the > message? > > If your specific project does not play nicely with this programme, > please let us know and we can add it to the list of exclusions on our > KDE Community page. > > I need your input, > - Kye Potter, KDE Gardening >
Re: RFC: an improved ki18n_install
El dilluns, 6 de març de 2023, a les 20:31:01 (CET), Milian Wolff va escriure:
> Hey all,
>
> for context please see [1] and [2].
>
> [1]: https://bugs.kde.org/show_bug.cgi?id=460245
> [2]:
> https://discourse.cmake.org/t/feature-request-add-custom-command-all/7609
>
> The gist is that me and Igor are annoyed by `ki18n_install` abusing
> `add_custom_target`, which makes the build always dirty. I.e. every time we
> run `ninja` we will, at the very least, run the two mo & ts generation
> targets. For kdevelop, these do non-trivial amounts of work that takes time
> even on a beefy laptop:
>
> ```
> $ ninja && time ninja
> [2/2] Generating mo...
> [2/2] Generating mo...
>
> real0m1,780s
> user0m5,742s
> sys 0m2,327s
> ```
>
> I would like to fix this, but first want to get feedback from the KDE
> developers at large.
>
> First of all: Are all projects using ki18n_install in the way we do? Why is
> no-one else complaining about this so far? Are your po files so small that
> this doesn't take a significant amount of time? Or are there potentially
> just so few of them in your project? KDevelop is heavily plugin based which
> means we have tons of po files. 2464 *.po files to be exact...
>
> Secondly: does anyone know how to best approach a solution to this issue?
> The problem is that I don't see an easy way to solve it: While Kyle Edwards
> was nice enough to show me a way to tell CMake to not make the custom
> target always-dirty, `k18n_install` suffers from another design deficiency:
> It uses globbing internally and has an undefined amount of inputs and
> outputs, which basically makes it impossible for us to leverage
> `add_custom_command(OUTPUT` here, or?
>
> As such, it seems like one would need a different approach to this problem
> anyways. Globbing is a known-evil in cmake land, and I would personally
> prefer to have the inputs explicitly listed, just like we do for source
> files etc. Because we have many languages though, listing all possible *.po
> or *.ts files is cumbersome. Instead, what about we maintain the list of
> known languages in the ki18n framework? Then we could have something like
>
> ```
> ki18n_target(
> # the target for which we are handling messages
> TARGET KF5I18n
> # the translation domain, added as compile_options and to find files
> TRANSLATION_DOMAIN ki18n5
> # the root po folder location, to look for the .po/.js files
> PO_FOLDER ${KI18n_SOURCE_DIR}/po
> )
> ```
>
> Internally, this would do what is currently done manually:
>
> ```
> target_compile_options(KF5I18n PRIVATE -DTRANSLATION_DOMAIN=\"ki18n5\")
> ```
>
> Then it would iterate over the list of known languages and try to find the
> .po or .js file. For every match, we would add_custom_target to generate
> the corresponding .mo/.ts file and add the reverse dependency.
>
> What do others say to this approach?
How is globbing (checking what is in the filesystem) different from the
checking
against a known list that will check against the filesystem if a file exists?
Seems the same thing but with extra steps to me.
Cheers,
Albert
>
> Thanks
Re: RFC: an improved ki18n_install
On Mon, Mar 6, 2023 at 8:31 PM Milian Wolff wrote:
>
> Hey all,
>
> for context please see [1] and [2].
>
> [1]: https://bugs.kde.org/show_bug.cgi?id=460245
> [2]: https://discourse.cmake.org/t/feature-request-add-custom-command-all/7609
>
> The gist is that me and Igor are annoyed by `ki18n_install` abusing
> `add_custom_target`, which makes the build always dirty. I.e. every time we
> run `ninja` we will, at the very least, run the two mo & ts generation
> targets. For kdevelop, these do non-trivial amounts of work that takes time
> even on a beefy laptop:
>
> ```
> $ ninja && time ninja
> [2/2] Generating mo...
> [2/2] Generating mo...
>
> real0m1,780s
> user0m5,742s
> sys 0m2,327s
> ```
>
> I would like to fix this, but first want to get feedback from the KDE
> developers at large.
>
> First of all: Are all projects using ki18n_install in the way we do? Why is
> no-one else complaining about this so far? Are your po files so small that
> this doesn't take a significant amount of time? Or are there potentially just
> so few of them in your project? KDevelop is heavily plugin based which means
> we have tons of po files. 2464 *.po files to be exact...
>
> Secondly: does anyone know how to best approach a solution to this issue? The
> problem is that I don't see an easy way to solve it: While Kyle Edwards was
> nice enough to show me a way to tell CMake to not make the custom target
> always-dirty, `k18n_install` suffers from another design deficiency: It uses
> globbing internally and has an undefined amount of inputs and outputs, which
> basically makes it impossible for us to leverage `add_custom_command(OUTPUT`
> here, or?
>
> As such, it seems like one would need a different approach to this problem
> anyways. Globbing is a known-evil in cmake land, and I would personally prefer
> to have the inputs explicitly listed, just like we do for source files etc.
> Because we have many languages though, listing all possible *.po or *.ts files
> is cumbersome. Instead, what about we maintain the list of known languages in
> the ki18n framework? Then we could have something like
>
> ```
> ki18n_target(
> # the target for which we are handling messages
> TARGET KF5I18n
> # the translation domain, added as compile_options and to find files
> TRANSLATION_DOMAIN ki18n5
> # the root po folder location, to look for the .po/.js files
> PO_FOLDER ${KI18n_SOURCE_DIR}/po
> )
> ```
>
> Internally, this would do what is currently done manually:
>
> ```
> target_compile_options(KF5I18n PRIVATE -DTRANSLATION_DOMAIN=\"ki18n5\")
> ```
>
> Then it would iterate over the list of known languages and try to find the .po
> or .js file. For every match, we would add_custom_target to generate the
> corresponding .mo/.ts file and add the reverse dependency.
>
> What do others say to this approach?
>
> Thanks
>
> --
> Milian Wolff
> http://milianw.de
>
>
+1 to getting the problem solved. I've also been bothered by this and
thought about looking into this so thanks for stepping up! (also I'm
pretty sure this code is originally mine from a time when we didn't
generate translations on every single build ^^').
Having ninja/make do this dependency generation can end up being more
work than worth it because then we need to have a static list and then
we need to re-run it when the po files change and it's all a mess. How
about changing build-pofiles/build-tsfiles to only execute the process
if the origin file is newer than the target?
I've put together a MR to explain what I meant (and I guess out of
guilt for having produced this :D)
https://invent.kde.org/frameworks/ki18n/-/merge_requests/81
Aleix
