public inbox for goredo-devel@lists.cypherpunks.ru
Atom feed
From: Sergey Matveev <stargrave@stargrave•org>
To: goredo-devel@lists.cypherpunks.ru
Subject: Re: redoing unnecessary targets when a do file is modified but the output remains unchanged
Date: Wed, 5 May 2021 18:11:02 +0300 [thread overview]
Message-ID: <YJK1lhMOV00DWBZr@stargrave.org> (raw)
In-Reply-To: <CAN5FJ7cy+zgjvU6o2BKE_+QDVr=RPA4Z0cE=nmWxwi7_dHG7Fg@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4985 bytes --]
*** Andrey Dobrovolsky [2021-05-05 16:52]:
>>redo just:
>> determines that A is OOD, because B is OOD, and then
>> executes A.do,
>
>In my opinion this is the origin of the problem, named in the message subject.
>In my implementation if A depends on B and B is outdated, then B.do is
>executed, then B is checked with the hash, and if it is found to be
>changed, only after this A.do is executed.
I understand that. And exactly that fact I do not like, that A.do
explicitly says "redo (current target A) if B one is changed" and
out-of-date algorithm decided that B is OOD -- so A is by definiton
(in my opinion) of "redo-ifchange B", has to be rebuild (and possibly
"B" dependency will be "thrown out" at all (no "redo-ifchange B" will
be called in A.do for some reason)).
>I really expect the issue, pointed in the message subject, very
>harmful and suppose that it can have negative influence on the redo
>system wide use.
I do not see how it makes redo less appealing that any kind of
Makefiles. Even native redo-c is completely worth of replacing
any kind of Make.
As I understand, the only problems with its algorithm may be with
redo-always targets. I met them only with that kind of targets. And
goredo contains an ugly hack, very similar to your algorithm: find if
any of dependencies has always-ed ones, execute them, then do like
redo-c does (excluding always-ed targets in OOD detection if their
hashes are same). So it is actually you suggestion: to run dependency
.do file first. Like automated:
redo some/always/target/that/is/a/dependency/of/some/real/target
redo -do-not-treat-always-as-ood-unless-checksum-differs some/real/target
And I made it because only of redo-always targets. Without redo-always
targets -- everything is fine and clear (redo-c native algorithm).
redo-always seems to be pain in the ass. Like with redo-stamp, found in
apenwarr/redo, in my opinion just gives huge unnecessary complexities
(hashing is cheap). DJB does not suggest redo-always at all: only
redo-ifchange, redo-ifcreate (necessary one) and redo. If you want to
run something all the time, use "redo" invocation on that target. And if
you want to redo only if dependencies are changes (probably that were
changed by previous "redo" invocation), then "redo-ifchange" them. So
you actually can "emulate" redo-always targets in native redo-c by
creating manually the .do-s with:
redo some/always ; redo-ifchange other/one
some/always will be built anyway, but redo-ifchange won't rebuild
other/one, if no checksum differs, after some/always invocation. Much
more simpler than adding redo-always command that... just uglifies very
simple and trivial redo-c algorithm. Inventing hard and complicated
things is easy: redo-stamp, redo-always. But creating simple .do files,
that makes them completely necessary -- is hard.
When I looked at various implementations source code (not deeply, just a
quick look), I thought that nearly everyone does the sample simple and
rock solid, determined algorithm like in redo-c.
As for goredo, again, it still has redo-always and redo-stamp, because
my pretty big projects originally were written "for" apenwarr/redo. And
I liked that if any of environment variables changes (hypothetical
$VERSION_PREFIX), then redo-always targets, dumping "env", making
redo-stamp, will OOD all the targets that will include software version
with that $VERSION_PREFIX. Then I made goredo and removed redo-stamp
from my .do files, because it is complexity and burdensome -- hashing is
very cheap. And redo-always behaviour can be replaced by "wrapper" .do-s
containing pure "redo" call, to some kind of dump-env.do, and
redo-ifchange calls further, depending on dump-env file.
Problem with the whole redo-ecosystem is that all of implementations
have different vision and opinion even on minor subjects. Someone just
literally executes executable .do-s. Others parses that .do-s to
determine is it shell-script, to add "-x" if necessary. Someone requires
shebangs to be present in .do-s, someone does not. Some create
.blablabla.target.name temporary files, some create .complete-random,
without any "target.name"-s extension that will possibly break some
commands expecting for known file extension (I met that kind of
annoynance when run redo-c on .do-s tested for apenwarr/redo). But all
of that is pretty trivial and small things and changes, unlike
redo-always (to make it "optimal" and not literally rebuilding the whole
project, because some envvar changed) and redo-stamp, influencing the
whole building algorithm of the redo itself. Now, after a year of redo
usage, I am convinced that redo/-ifchange/-ifcreate are completely
enough for nearly all the tasks. No redo-always -- no problems with
simple, clear, solid redo-c algorithm :-)
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263 6422 AE1A 8109 E498 57EF
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2021-05-05 15:11 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-05 13:52 redoing unnecessary targets when a do file is modified but the output remains unchanged Andrey Dobrovolsky
2021-05-05 15:11 ` Sergey Matveev [this message]
-- strict thread matches above, loose matches on Subject: below --
2021-05-05 18:35 Andrey Dobrovolsky
2021-05-04 22:52 Andrey Dobrovolsky
2021-05-05 7:35 ` Sergey Matveev
2021-02-17 20:40 Karolis K
2021-02-21 8:38 ` Sergey Matveev
2021-02-27 9:09 ` Sergey Matveev