public inbox for nncp-devel@lists.cypherpunks.ru Atom feed
* Suggestions to make ACK easier @ 2023-07-11 14:28 John Goerzen 2023-07-11 15:55 ` John Goerzen ` (2 more replies) 0 siblings, 3 replies; 7+ messages in thread From: John Goerzen @ 2023-07-11 14:28 UTC (permalink / raw) To: nncp-devel Hi, A question about ACKs came up on the Matrix channel today, which prompted me to revisit my previous article about it, at https://www.complete.org/dead-usb-drives-are-fine-building-a-reliable-sneakernet/ There are two sort of pain points: 1) There is a race condition in that nncp-ack must run before nncp-toss for incoming packets. 2) We have to save off the list of generated ACKs so that we can remove them from the queue later. I think the situation could be simplified with two tweaks: First, nncp-xfer (and all the others, nncp-call, caller, daemon, bundle, etc) could have a -ack mode that will cause it to automatically generate an ack after successful receipt of a packet, eliminating the nncp-toss race condition. (Or maybe it would make more sense to put this in toss? But there I think it would be more easy to generate multiple ACKs) Then, nncp-xfer -tx could have a -keepexceptacks or something, that would cause it to keep all packets it sends, EXCEPT for the acks. That eliminates the need to save off the list of generated ACKs from an earlier step. It may be that the status of a packet as an ACK isn't available at this point, in which case a separate flag file or something may be useful. Thanks, John ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Suggestions to make ACK easier 2023-07-11 14:28 Suggestions to make ACK easier John Goerzen @ 2023-07-11 15:55 ` John Goerzen 2023-07-30 7:47 ` Sergey Matveev 2023-08-09 12:04 ` Sergey Matveev 2 siblings, 0 replies; 7+ messages in thread From: John Goerzen @ 2023-07-11 15:55 UTC (permalink / raw) To: nncp-devel I guess one other thought - doing it at toss time, while it may have a risk of acking more than once, could also be nice as it would make it convenient to do end-to-end ACKs even on cases where things traverse several network hops. - John On Tue, Jul 11 2023, John Goerzen wrote: > Hi, > > A question about ACKs came up on the Matrix channel today, which > prompted me to revisit my previous article about it, at > https://www.complete.org/dead-usb-drives-are-fine-building-a-reliable-sneakernet/ > > There are two sort of pain points: > > 1) There is a race condition in that nncp-ack must run before nncp-toss > for incoming packets. > > 2) We have to save off the list of generated ACKs so that we can remove > them from the queue later. > > I think the situation could be simplified with two tweaks: > > First, nncp-xfer (and all the others, nncp-call, caller, daemon, bundle, > etc) could have a -ack mode that will cause it to automatically generate > an ack after successful receipt of a packet, eliminating the nncp-toss > race condition. (Or maybe it would make more sense to put this in toss? > But there I think it would be more easy to generate multiple ACKs) > > Then, nncp-xfer -tx could have a -keepexceptacks or something, that would > cause it to keep all packets it sends, EXCEPT for the acks. That > eliminates the need to save off the list of generated ACKs from an > earlier step. It may be that the status of a packet as an ACK isn't > available at this point, in which case a separate flag file or something > may be useful. > > Thanks, > > John ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Suggestions to make ACK easier 2023-07-11 14:28 Suggestions to make ACK easier John Goerzen 2023-07-11 15:55 ` John Goerzen @ 2023-07-30 7:47 ` Sergey Matveev 2023-08-09 12:04 ` Sergey Matveev 2 siblings, 0 replies; 7+ messages in thread From: Sergey Matveev @ 2023-07-30 7:47 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 428 bytes --] Greetings! *** John Goerzen [2023-07-11 09:28]: [...] Very sorry for late answer, but I still do not have enough time even to think about the suggestions. My vacation on work was lost and currently do not know exactly when I will have enough free time. Hoping the next month I will ne on the "rail" again! -- Sergey Matveev (http://www.stargrave.org/) OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Suggestions to make ACK easier 2023-07-11 14:28 Suggestions to make ACK easier John Goerzen 2023-07-11 15:55 ` John Goerzen 2023-07-30 7:47 ` Sergey Matveev @ 2023-08-09 12:04 ` Sergey Matveev 2023-08-10 2:04 ` John Goerzen 2 siblings, 1 reply; 7+ messages in thread From: Sergey Matveev @ 2023-08-09 12:04 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 1608 bytes --] Greetings! *** John Goerzen [2023-07-11 09:28]: >First, nncp-xfer (and all the others, nncp-call, caller, daemon, bundle, >etc) could have a -ack mode that will cause it to automatically generate >an ack after successful receipt of a packet, eliminating the nncp-toss >race condition. (Or maybe it would make more sense to put this in toss? >But there I think it would be more easy to generate multiple ACKs) I am afraid I dislike all those ideas unfortunately, basically because of non-atomic operations. If "nncp-xfer -rx" will generate ACK packet, then what if something will go wrong and outbound ACK packet could not be saved? We loose that ACK and our running nncp-toss can already toss retrieved packet. -xfer + -ack + -toss guarantees that if -ack will fail, then no packet will disappear because of tossing. We can rerun -ack and be sure that ACKs are generated (or will catch some error again (missing free disk space for example)), and only after that we will toss them. So that won't eliminate the race between reception and tossing. >Then, nncp-xfer -tx could have a -keepexceptacks or something, that would >cause it to keep all packets it sends, EXCEPT for the acks. Unfortunately it can not, because -xfer sees only already encrypted packets, which lacks packet type identification. It does not know what it transfers. Or we have to keep some state, list of encrypted ACK packets that can be safely removed, that leads to current workflow again. -- Sergey Matveev (http://www.stargrave.org/) OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Suggestions to make ACK easier 2023-08-09 12:04 ` Sergey Matveev @ 2023-08-10 2:04 ` John Goerzen 2023-08-13 20:30 ` Sergey Matveev 0 siblings, 1 reply; 7+ messages in thread From: John Goerzen @ 2023-08-10 2:04 UTC (permalink / raw) To: Sergey Matveev; +Cc: nncp-devel On Wed, Aug 09 2023, Sergey Matveev wrote: > [[PGP Signed Part:Undecided]] > Greetings! > > *** John Goerzen [2023-07-11 09:28]: >>First, nncp-xfer (and all the others, nncp-call, caller, daemon, bundle, >>etc) could have a -ack mode that will cause it to automatically generate >>an ack after successful receipt of a packet, eliminating the nncp-toss >>race condition. (Or maybe it would make more sense to put this in toss? >>But there I think it would be more easy to generate multiple ACKs) > > I am afraid I dislike all those ideas unfortunately, basically because > of non-atomic operations. > > If "nncp-xfer -rx" will generate ACK packet, then what if something will > go wrong and outbound ACK packet could not be saved? We loose that ACK > and our running nncp-toss can already toss retrieved packet. -xfer + That makes sense. Though if -seen is in use, I guess it will just cause a harmless retransmission. > -ack + -toss guarantees that if -ack will fail, then no packet will > disappear because of tossing. We can rerun -ack and be sure that ACKs But there is the problem that toss may run before -ack. Basically if anything at all on the system is going to use -ack, then you can't use -autotoss, or nncp-toss with -cycle, because that then means a race condition -- packets may be tossed before nncp-ack is run. You basically have to ensure that nothing at all might run toss when you're loading stuff up for -ack, and that means some outside-of-NNCP locking in some cases. Maybe the thing to do here is extend nncp.hjson to add an "autoack" feature to a given node? That way, nncp-toss could generate an ack for it at toss time. This may imply needing a seen-like method to prevent sending multiple ACKs for a packet that fails to toss the first time. One other thing I've been pondering: end-to-end ACKs. So if you send from node A to node D via B and C, what happens if B receives the packet and then its disk dies? This is similar to the USB drive case. I suppose we could have each piece along the chain generate ACKs to the previous one, or could we have D generate an ack back to A when the packet arrives at D? That may be too complicated, but just a thought I've been having. > are generated (or will catch some error again (missing free disk space > for example)), and only after that we will toss them. So that won't > eliminate the race between reception and tossing. > >>Then, nncp-xfer -tx could have a -keepexceptacks or something, that would >>cause it to keep all packets it sends, EXCEPT for the acks. > > Unfortunately it can not, because -xfer sees only already encrypted > packets, which lacks packet type identification. It does not know what > it transfers. Or we have to keep some state, list of encrypted ACK > packets that can be safely removed, that leads to current workflow again. Ahh, OK, that makes sense. Thanks, - John ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Suggestions to make ACK easier 2023-08-10 2:04 ` John Goerzen @ 2023-08-13 20:30 ` Sergey Matveev 2023-09-13 17:29 ` John Goerzen 0 siblings, 1 reply; 7+ messages in thread From: Sergey Matveev @ 2023-08-13 20:30 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 2547 bytes --] Greetings! *** John Goerzen [2023-08-09 21:04]: >> If "nncp-xfer -rx" will generate ACK packet, then what if something will >> go wrong and outbound ACK packet could not be saved? We loose that ACK >> and our running nncp-toss can already toss retrieved packet. -xfer + > >That makes sense. Though if -seen is in use, I guess it will just cause >a harmless retransmission. Problem here is that ACK won't be generated at all, because packet is already tossed. Duplicate ACKs is not a problem indeed, but completely lost (non-generated) ones are bad of course. >But there is the problem that toss may run before -ack. Basically if >anything at all on the system is going to use -ack, then you can't use >-autotoss, or nncp-toss with -cycle, because that then means a race >condition -- packets may be tossed before nncp-ack is run. You >basically have to ensure that nothing at all might run toss when you're >loading stuff up for -ack, and that means some outside-of-NNCP locking >in some cases. Everything is right. ACK-based workflow was not compatible with autotossing. >Maybe the thing to do here is extend nncp.hjson to add an "autoack" >feature to a given node? That way, nncp-toss could generate an ack for >it at toss time. As with -xfer/-bundle commands, you have to store the knowledge of packets who carry ACKs. It was made by printing their identifiers to stdout by nncp-ack command itself. With tossers (that are not only -toss, but -daemon and -caller) you will have to create some kind of atomically commitable database of those ACKs. It can be a directory with just empty files, whose names will be ACK-packets identifiers. I made ability to automatically generate ACKs during tossing in 8.9.0 release. nncp-toss -gen-ack, nncp-* -autotoss-gen-ack. Although I did only quick testing if it works fine. It saves knowledge about ACK packets in spool/tx/ack/PKT empty files. nncp-rm -ack will remove them and corresponding spool/tx/PKT packets. >One other thing I've been pondering: end-to-end ACKs. So if you send >from node A to node D via B and C, what happens if B receives the packet >and then its disk dies? Something badly unexpected will happen :-) Well, end-to-end acknowledgments is more complicated, because recipient's node does not know identity of the very first packet that was sent. Something much more complicated must be done to deal with it. -- Sergey Matveev (http://www.stargrave.org/) OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Suggestions to make ACK easier 2023-08-13 20:30 ` Sergey Matveev @ 2023-09-13 17:29 ` John Goerzen 0 siblings, 0 replies; 7+ messages in thread From: John Goerzen @ 2023-09-13 17:29 UTC (permalink / raw) To: Sergey Matveev; +Cc: nncp-devel On Sun, Aug 13 2023, Sergey Matveev wrote: > I made ability to automatically generate ACKs during tossing in 8.9.0 > release. nncp-toss -gen-ack, nncp-* -autotoss-gen-ack. Although I did > only quick testing if it works fine. It saves knowledge about ACK > packets in spool/tx/ack/PKT empty files. nncp-rm -ack will remove them > and corresponding spool/tx/PKT packets. Hi Sergey, Thank you for this! It's been a busy month, but I've finally had the time to update my documentation, scripts, and such, and test this out. I think I've found two issues. The first, if -gen-ack is used in conjunction with -seen, then -seen fires first and the ACK isn't generated. This is relevant if the ACK itself was lost at some point; we'd want to keep generating them as long as the relevant packet keeps coming in. Ideally the ACK would be processed before applying the -seen filter. The second, I'm a little less sure of, but it seems that when responding to packets that arrived using a via designation, the ACK is sent to the wrong node -- not the immediate peer. I had a situation where a packet flowed like this: A -> Z -> V where the path between Z and V is using a USB stick and runs ACK. The ACK generated on V might have been sent to A via Z, rather than to Z directly. I'm a little unsure of my diagnosis here; I did see A process an ACK, but it is also theoretically possible that a nncp-toss -gen-ack running on Z generated that ACK. In any case, somehow there were a few packets that don't have ACKs and are sticking around (hence the first issue). Thanks! - John ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2023-09-13 17:33 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-07-11 14:28 Suggestions to make ACK easier John Goerzen 2023-07-11 15:55 ` John Goerzen 2023-07-30 7:47 ` Sergey Matveev 2023-08-09 12:04 ` Sergey Matveev 2023-08-10 2:04 ` John Goerzen 2023-08-13 20:30 ` Sergey Matveev 2023-09-13 17:29 ` John Goerzen