public inbox for goredo-devel@lists.cypherpunks.ru
Atom feed
* Nested dirs for non existing targets
@ 2022-04-14 2:05 Andrew Chambers
2022-04-14 7:18 ` goredo
2022-04-15 16:22 ` Sergey Matveev
0 siblings, 2 replies; 9+ messages in thread
From: Andrew Chambers @ 2022-04-14 2:05 UTC (permalink / raw)
To: goredo-devel
Hi there,
I noticed every time I mistype a target, I have to delete the
directories that it creates... e.g.
If I mean to type:
redo ../foo/bar/baz
and I accidentally type:
redo ./foo/bar/baz
I will need to clean up some junk directories that goredo has created.
Other redo implementations do not seem to create nested target
directories like this, is this the intended behavior? I wondered if
perhaps it only meant to create the .redo directory, but not necessarily
the parents that do not exist. I wonder if it should avoid creating the
empty target directories if they do not exist and instead raise an error.
Regards,
Andrew Chambers
^ permalink raw reply [flat|nested] 9+ messages in thread
* Nested dirs for non existing targets
2022-04-14 2:05 Nested dirs for non existing targets Andrew Chambers
@ 2022-04-14 7:18 ` goredo
2022-04-14 11:01 ` Andrew Chambers
2022-04-15 16:22 ` Sergey Matveev
1 sibling, 1 reply; 9+ messages in thread
From: goredo @ 2022-04-14 7:18 UTC (permalink / raw)
To: goredo-devel
Short answer: Yes, that is the intended behavior. Goredo maintains a distributed database and has to make an entry for your mistyped target at the target directory.
Every time redo identifies at least the .do file it uses to try and produce your mistyped target, it needs to keep record of it.
Regards,
–Michael
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 7:18 ` goredo
@ 2022-04-14 11:01 ` Andrew Chambers
2022-04-14 11:30 ` Andrew Chambers
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Chambers @ 2022-04-14 11:01 UTC (permalink / raw)
To: goredo-devel
> Short answer: Yes, that is the intended behavior. Goredo maintains a distributed database and has to make an entry for your mistyped target at the target directory.
I understand this is the case for targets that would be created in an
existing directory, but if there is no directory I am not sure why
goredo creates it. This seems like an incompatibility with other redo
implementations.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 11:01 ` Andrew Chambers
@ 2022-04-14 11:30 ` Andrew Chambers
2022-04-14 12:01 ` goredo
2022-04-15 16:23 ` Sergey Matveev
0 siblings, 2 replies; 9+ messages in thread
From: Andrew Chambers @ 2022-04-14 11:30 UTC (permalink / raw)
To: goredo-devel
>> Short answer: Yes, that is the intended behavior. Goredo maintains a
>> distributed database and has to make an entry for your mistyped
>> target at the target directory.
>
>
> I understand this is the case for targets that would be created in an
> existing directory, but if there is no directory I am not sure why
> goredo creates it. This seems like an incompatibility with other redo
> implementations.
>
>
Ah, I see my mistake now - in some ways this is perhaps somewhat
unfortunate, though I'm not sure how to avoid it. One way would be for
goredo to store the dependency information in the directory containing
the do file instead of the directory containing the target.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 11:30 ` Andrew Chambers
@ 2022-04-14 12:01 ` goredo
2022-04-14 12:05 ` Andrew Chambers
2022-04-15 16:23 ` Sergey Matveev
1 sibling, 1 reply; 9+ messages in thread
From: goredo @ 2022-04-14 12:01 UTC (permalink / raw)
To: goredo-devel
True, apenwarr/redo does that, but this has other drawbacks, as well. Storing the database in a central place breaks composability and has weird effects with directory symlinks.
Also, because a repository root is not a very well-defined concept, it is generally unclear where to put the database in the first place.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 12:01 ` goredo
@ 2022-04-14 12:05 ` Andrew Chambers
2022-04-14 14:29 ` goredo
0 siblings, 1 reply; 9+ messages in thread
From: Andrew Chambers @ 2022-04-14 12:05 UTC (permalink / raw)
To: goredo-devel
> True, apenwarr/redo does that, but this has other drawbacks, as well. Storing the database in a central place breaks composability and has weird effects with directory symlinks.
>
> Also, because a repository root is not a very well-defined concept, it is generally unclear where to put the database in the first place.
>
I think apenwarr/redo tries to define a repository root, which is a big
mistake. I was proposing storing the dep info next to the .do file which
is a different thing.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 12:05 ` Andrew Chambers
@ 2022-04-14 14:29 ` goredo
0 siblings, 0 replies; 9+ messages in thread
From: goredo @ 2022-04-14 14:29 UTC (permalink / raw)
To: goredo-devel
That doesn't work either. Example:
/default.do is used to produce /a/b/file, which fails but records an entry in /.redo
Later, you create /a/default.do and build /a/b/file again. You create a record in /a/.redo and are left with a bogus record in /.redo. Not good!
You can never look at a piece of the database and expect the records to be valid. You would always have to traverse all record up-/downstream which is largely equivalent to a central database with extra steps.
You want either a database in a single directory or you want it tied to the target, not the .do file. Both have benefits and drawbacks.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 11:30 ` Andrew Chambers
2022-04-14 12:01 ` goredo
@ 2022-04-15 16:23 ` Sergey Matveev
1 sibling, 0 replies; 9+ messages in thread
From: Sergey Matveev @ 2022-04-15 16:23 UTC (permalink / raw)
To: goredo-devel
[-- Attachment #1: Type: text/plain, Size: 1764 bytes --]
*** Andrew Chambers [2022-04-14 23:30]:
>One way would be for
>goredo to store the dependency information in the directory containing the
>do file instead of the directory containing the target.
That heavily complicates everything. If you have dependency information
stored near the target itself, then finding it is trivial:
for foo/bar/baz/TGT
dependency information is in foo/bar/baz/.redo/TGT.rec
But if you store it "near" the .do file, then:
for foo/bar/baz/TGT
.do file is: foo/default.do,
so depinfo is in: foo/.redo/bar_baz_TGT.rec (for example)
and for finding it, you have to traverse from foo/bar/baz/ directory to
uppper levels finding that bar_baz_TGT.rec file.
And you can not just call find_do_file(foo/bar/baz/TGT) and go to that
directory, because another .do file can appear after that.
foo/bar/default.do appeared
and it should be the default one for foo/bar/baz/TGT
but still only foo/.redo/bar_baz_TGT.rec has dependency information
And when you rerun your target, foo/bar/.redo/baz_TGT.rec will appear...
but what to do with the left foo/.redo/bar_baz_TGT.rec? Search for
possible obsolete dependency information and delete it? It will be
impossible to do that atomically: you can not rename temporary file to
foo/bar/.redo/baz_TGT.rec and delete foo/.redo/bar_baz_TGT.rec at once.
That leads to the desire of some centralized storage, as apenwarr/redo
did. But its author decided that it was a bad idea:
https://redo.readthedocs.io/en/latest/FAQImpl/#how-does-redo-store-dependencies
Well, spacefrogg@ actually wrote about exactly all of that too.
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263 6422 AE1A 8109 E498 57EF
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Nested dirs for non existing targets
2022-04-14 2:05 Nested dirs for non existing targets Andrew Chambers
2022-04-14 7:18 ` goredo
@ 2022-04-15 16:22 ` Sergey Matveev
1 sibling, 0 replies; 9+ messages in thread
From: Sergey Matveev @ 2022-04-15 16:22 UTC (permalink / raw)
To: goredo-devel
[-- Attachment #1: Type: text/plain, Size: 644 bytes --]
Greetings!
*** Andrew Chambers [2022-04-14 14:05]:
>Other redo implementations do not seem to create nested target directories
>like this, is this the intended behavior? I wondered if perhaps it only
>meant to create the .redo directory
It was not explicitly intended to do that (to behave like "mkdir -p"),
but you are right that it is just a side-effect of creating .redo
subdirectory with temporary file containing redo-ifcreate dependencies
recorded during the search for .do file. As spacefrogg noticed correctly.
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263 6422 AE1A 8109 E498 57EF
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-04-15 16:23 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-14 2:05 Nested dirs for non existing targets Andrew Chambers
2022-04-14 7:18 ` goredo
2022-04-14 11:01 ` Andrew Chambers
2022-04-14 11:30 ` Andrew Chambers
2022-04-14 12:01 ` goredo
2022-04-14 12:05 ` Andrew Chambers
2022-04-14 14:29 ` goredo
2022-04-15 16:23 ` Sergey Matveev
2022-04-15 16:22 ` Sergey Matveev