public inbox for goredo-devel@lists.cypherpunks.ru
Atom feed
From: Sergey Matveev <stargrave@stargrave•org>
To: goredo-devel@lists.cypherpunks.ru
Subject: Re: redo-ifchange hangs when called with many arguments
Date: Tue, 19 Oct 2021 16:58:41 +0300	[thread overview]
Message-ID: <YW7PEbyx1Pf+7bTG@stargrave.org> (raw)
In-Reply-To: <f45ec2e1-3c62-4f5a-8e13-22e3006dde7e@spacefrogg.net>

[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]

Greetings!

*** goredo [2021-10-16 15:58]:
>One of my projects resulted in an inadvertent stress test of how many arguments redo-ifchange can handle.

Well, basically it should depend solely on OS. Of course except for
maximal command line arguments number, that is the main limiting factor
of redo. I wonder why DJB decided to pass dependencies as arguments (as
tar), instead of reading them from stdin (as cpio/pax), that would not
artificially limit their number.

>The redo-ifchange process was just sleeping without making progress, like it was waiting for pipe input or something similar.

I do not clearly remember (possibly I am mistaken), but I think I met
that kind of behaviour last year when I run several thousand targets at
once in parallel. I do not remember how I debugged and traced it, but
the problem was in touching limits of opened file descriptors, that I
fixed in 1.2.0 release (it was non-optimal resources usage, but everything
got stuck at OS level limits). I also noticed that my FreeBSD has nearly
two million opened files limit, but some version of Ubuntu just only one
thousand.

>I also noticed before, that killing redo with C-c does not always terminate the process it spawned (through a .do file).

According to the code (and I checked it in practice), it should not
terminate any of its processes at all :-). I added "infanticide" to it
in 1.17.0 release.

-- 
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263  6422 AE1A 8109 E498 57EF

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  reply	other threads:[~2021-10-19 13:58 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-16 15:58 redo-ifchange hangs when called with many arguments goredo
2021-10-19 13:58 ` Sergey Matveev [this message]
2021-10-19 14:56   ` goredo
2021-10-19 15:26     ` Sergey Matveev
2021-10-19 16:05       ` goredo
2021-10-21  9:51         ` Sergey Matveev