public inbox for goredo-devel@lists.cypherpunks.ru
Atom feed
* Several issues since 1.30
@ 2023-09-21 19:12 spacefrogg
  2023-09-23 19:16 ` resource deadlock avoided messages Sergey Matveev
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: spacefrogg @ 2023-09-21 19:12 UTC (permalink / raw)
  To: goredo-devel

Hi,

I am experiencing several issues since updating to 1.30 (maybe present 
one or two releases earlier, too).

1. I have a C project with a very flat hierarchy. About 90 C files are 
to be compiled and linked into 5 static libraries. The compiler command 
is pre-computed in a separate script and used (and redo-ifchange'd) in 
the default.o target that compiles each C file. On a fresh run which 
must establish the dependency the first time, I run into the following 
warning issued by redo:

     run.go:247: resource deadlock avoided 
/home/user/project/out/.redo/cc.precompile.lock

N.B. The particular message shown above refers to the precompile script 
that generates precompiled header files. It bears the same dependency 
relation as the compile script: Each C file depends on it. Sometimes it 
hits at that target, sometimes it hits the compiler script. This warning 
is issued about 320 times. So, many times more than files depending on 
it.

To be clear, everything works as expected, but it seems that redo has 
become greedy of sorts and exhausts some of its resources. My toolchain 
does not limit the amount of targets discharged to a `redo-ifchange` 
call. So, I suspect the number of open files is an issue.

2. Sometimes, again for projects with many independent targets that can 
run in parallel, redo *at least* prints out that it ran one of those 
targets twice that all the targets depended upon (e.g. the compile 
script). I suspect a serious issue here that might relate to no. 1. 
Maybe, sometimes redo looses track of whether a target's dependency is 
already handled due to the resource exhaustion. I could not verify 
whether redo actually runs the target twice or just prints out the line 
a second time. The two status lines are not always adjacent to each 
other (some other status line(s) might be in between), which, to me, 
hints to that it is actually executed multiple times.

3. The output of redo has become useless for larger amounts of targets 
that can run in parallel. Note, that this effect is independent of 
whether I run redo with a single job or multiple parallel jobs. What 
happens, is, that the output of each target is printed one after another 
without properly inserting the separators that conclude the run of each 
target (e.g. redo . . yadda/yadda (yadda.do) (10.123s)). All the missing 
separators are printed in a batch at some point. So, the general order 
of things, i.e., log output first, separator line second, is preserved. 
I suspect that some terminal flow control/buffering and stream handling 
has changed internally and that this is a race condition between the log 
output handler and the status printing (job manager?) handler. I also 
noticed that sometimes, redo does not print any separators for quite a 
while (while counting up the executed targets) and then printing many at 
once at some point. I noticed this for targets that do not produce any 
output. I suspect that this effect has the same cause.

4. In light of no. 3, I have a small feature request: The output of 
targets can be very confusing, because it is unclear at first, what 
target the output relates to. It just starts printing and only tells you 
in the very last line, which target actually produced the output. 
apenwarr redo does a better job, here, by printing two similar status 
lines, once as a header and once as a footer. So the feature request is 
as follows: As long as a target produces no output, do not print 
anything. If it starts producing output, print a header and then its 
output. Always print a footer after the end of a target. So, as long as 
targets produce no output, the list of completed targets looks the same 
as now, but targets that produce output are properly introduced by a 
header line for better context.

Thanks for reading this far!
–Michael

^ permalink raw reply	[flat|nested] 12+ messages in thread
end of thread, other threads:[~2023-10-16 13:24 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-21 19:12 Several issues since 1.30 spacefrogg
2023-09-23 19:16 ` resource deadlock avoided messages Sergey Matveev
2023-09-24 11:17   ` goredo
2023-09-25 16:14     ` Sergey Matveev
2023-09-23 19:16 ` Target runs multiple times in single session Sergey Matveev
2023-09-23 19:17 ` Slow/buffered output of targets finished messages Sergey Matveev
2023-09-23 19:17 ` Unclear what target the output relates to Sergey Matveev
2023-09-24 11:10   ` goredo
2023-09-25 16:20     ` Sergey Matveev
2023-09-28 13:55       ` spacefrogg
2023-09-30 16:57         ` Sergey Matveev
2023-10-16 13:24         ` Sergey Matveev