public inbox for goredo-devel@lists.cypherpunks.ru
Atom feed
* Re: redoing unnecessary targets when a do file is modified but the output remains unchanged
@ 2021-05-04 22:52 Andrey Dobrovolsky
  2021-05-05  7:35 ` Sergey Matveev
  0 siblings, 1 reply; 8+ messages in thread
From: Andrey Dobrovolsky @ 2021-05-04 22:52 UTC (permalink / raw)
  To: goredo-devel

Hi!

I was fascinated by redo idea after opennet.ru announced Your goredo
release, Your habr article helped a lot too, thanks for both!
Now I use my fork of Leah Neukirchen's redo-c. The problem You are
talking about bothered me too, and I've solved it in my  dev2 branch
at github.com/AndreyDobrovolskyOdessa/redo-c. This branch is for my
personal use, I am "noname" programmer and am not much interested
(yet?) in multi-threaded redoing, that's why current version is
single-threaded. But hashed dependencies allow to do only what is
really needed to be done, and that's really great!
The idea is, that while dependencies description allows, we can
recurse down the dependency tree until we will find something, that
really is to be done, and then we continue execution in down-to-top
manner, checking hashes at every node. This took full rewriting, but I
am satisfied with the result, and now consider the current version
utilizing full power of hashed dependencies.
I will be glad if my solution will help You with Your excellent project.

Regards!
Andrey Dobrovolsky

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: redoing unnecessary targets when a do file is modified but the output remains unchanged
@ 2021-05-05 18:35 Andrey Dobrovolsky
  0 siblings, 0 replies; 8+ messages in thread
From: Andrey Dobrovolsky @ 2021-05-05 18:35 UTC (permalink / raw)
  To: goredo-devel

Thank You for the very comprehensive answer.

Best regards!
Andrey Dobrovolsky

^ permalink raw reply	[flat|nested] 8+ messages in thread
* Re: redoing unnecessary targets when a do file is modified but the output remains unchanged
@ 2021-05-05 13:52 Andrey Dobrovolsky
  2021-05-05 15:11 ` Sergey Matveev
  0 siblings, 1 reply; 8+ messages in thread
From: Andrey Dobrovolsky @ 2021-05-05 13:52 UTC (permalink / raw)
  To: goredo-devel

Hello!

Thanks for paying attention to my message. I'm sorry that probably it
was formulated in a wrong way, and the main meaning was not expressed
sufficiently clear. I was not pointing at my implementation as at the
reference one, I meant that dependencies processing logic, that I
used, allows to avoid unnecessary .do operations. Exactly in
correspondence with the message subject. Let me, please, take one more
attempt.

So, according to
http://lists.cypherpunks.ru/archive/goredo-devel/2102/0015.html:
>redo just:
>    determines that A is OOD, because B is OOD, and then
>    executes A.do,

In my opinion this is the origin of the problem, named in the message subject.
In my implementation if A depends on B and B is outdated, then B.do is
executed, then B is checked with the hash, and if it is found to be
changed, only after this A.do is executed.
Technically program flow may be described schematically as:

redo_target(SomeTarget)
{
    if(SomeTarget has no .do file)
        return ok;
    while(TheDep = read(DepOfSomeTarget)) {
        if(redo_target(TheDep) is not ok)
            break;
        if(CompareHash(TheDep, HashFromDepOfSomeTarget) != ok)
            break;
    }
    if(not all deps of SomeTarget are ok)
        execute(SomeTarget.do);
}

In other words if A depends on B, one can make the decision on A being
OOD only after B will be updated.

This is the main difference of my version from the original Leah
Neukirchen's, other differences are consequences and some are
implementation details, which are not important and in no way are
promoted. As I can understand, Your implementation follows the same
logic.

 You can easily compile and check my redo version on the dep trees,
that seem to be problematic, "redo" is not equivalent to
"redo-always", this was wrong impression. The only confusing may be
that alone "redo" will not build "all" target, targets must be named
directly.

I really expect the issue, pointed in the message subject, very
harmful and suppose that it can have negative influence on the redo
system wide use.

Best regards!
Andrey Dobrovolsky

^ permalink raw reply	[flat|nested] 8+ messages in thread
* redoing unnecessary targets when a do file is modified but the output remains unchanged
@ 2021-02-17 20:40 Karolis K
  2021-02-21  8:38 ` Sergey Matveev
  2021-02-27  9:09 ` Sergey Matveev
  0 siblings, 2 replies; 8+ messages in thread
From: Karolis K @ 2021-02-17 20:40 UTC (permalink / raw)
  To: goredo-devel

Hello,

I recently encountered a behaviour that seems to be sub-optimal.

Consider a simple example with 2 do files:

A.txt.do:

echo “some text here”

B.txt.do:

redo-ifchange A.txt
cat A.txt | tr [:lower:] [:upper:]

After calling redo B.txt the targets are produced.

Now consider a cosmetic change to A.txt.do - for example - adding an empty line at the end of the file.
First - redo-ood will show that both A.txt and B.txt are out of date.
Then, if I redo-ifchange A.txt and call redo-ood after that - B.txt is no longer presented as being out of date.
However, if instead of the above I call redo-ifchange B.txt the file B.txt will also be rebuilt, even thou none of it’s direct dependencies changed.

This is of course a simple toy example, in one real project I encountered a situation where multiple targets were being rebuilt multiple times, all after adding an empty line to one of the do scripts.

In an ideal case I think only the file with a modified do script should be rebuilt, and then, since the output is the same, all other dependencies should simply pass.

But maybe I am missing something?

Thanks a lot for this redo implementation,
Kind regards,
Karolis K.

^ permalink raw reply	[flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-05-05 18:38 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-04 22:52 redoing unnecessary targets when a do file is modified but the output remains unchanged Andrey Dobrovolsky
2021-05-05  7:35 ` Sergey Matveev
  -- strict thread matches above, loose matches on Subject: below --
2021-05-05 18:35 Andrey Dobrovolsky
2021-05-05 13:52 Andrey Dobrovolsky
2021-05-05 15:11 ` Sergey Matveev
2021-02-17 20:40 Karolis K
2021-02-21  8:38 ` Sergey Matveev
2021-02-27  9:09 ` Sergey Matveev