*** John Goerzen [2022-03-02 10:18]:
>In theory, we sort of have two options:
>
>1) Never keep ACK packets on the source machine, and never send ACKs for
>them on the destination.
>
>2) Handle them like any other packet.  But this implies an endless
>stream of re-ACKing.

I honestly forget about an endless reACKing problem. Somehow we must not
send ACK for an ACK obviously. Currently nncp-ack is a separate command
call, that just generates ACK for everything it saw in rx-directory.
nncp-toss is a candidate for that job instead, because it sees the plain
(unencrypted) packet's type. But that will lead to more configuration
file's options to control if ACKs need to be generated. However,
nncp-ack already has access to private keys (that is required to
generate outbound encrypted packets), so it also can decrypt packets.

>That implies two questions then: how to prevent nncp-ack from itself
>ACKing an ACK, and how to cause nncp-xfer to not keep ACKs?

First part is solved by partly decrypting the packet in rx directory. We
need to decrypt only its first chunk, to view the plain packet's header
to determine the type. Basically it requires opening of each file,
reading up to 128+KiBs of data and two asymmetric cryptography
operations. I think that is acceptable.

Second part currently is not solvable. Because all encrypted packets are
indistinguishable from the noise, except for some header information,
with niceness level, that actually is not directly related to payload
type. I did not want to add visible packet's type outside the ciphertext
very much. nncp-xfer does not have access to private keys by design.

For example there can be an additional state which keeps knowledge of
ACK packets identifiers. Even empty "PKT.ack" nearby the "PKT" in the
spool area would satisfy that, but that means leaking of exactly that
special packet type knowledge. Currently it can be done for example by
enabling -debug logging during nncp-ack, parsing output recfile to get
generated ACK packets filenames. Then it can be used to remove those
packets after the transfer for example. I agree that lost ACK is not a
big problem. nncp-ack could have something like:
    nncp-ack -list-of-generated-acks acks.txt ...
    nncp-xfer / whatever
    nncp-rm -list-of-generated-acks acks.txt
That file can be placed in memory, because of the small size.

>I *THINK* we can do this with niceness.

Probably, but I do not like that idea at all. Niceness is not a "type of
the packet", however it is tempting to use it for that. Currently there
are different predefined default niceness levels for each type of
packet, but user is free to override it in anyway, for example to
completely remove any difference between all packets at all (of course
loosing ability to prioritize the packets).

>Somewhat related - I have nncp-toss running as a daemon (-cycle) mode on
>some nodes that I will want to enable ack generation for.

I tend to treat "nncp-toss -cycle" like a mistake done previously. It
complicates things very much, because of concurrent access, because
every option is have to be in configuration file, instead of
simpler-to-process command line arguments. But ok, -cycle exists, I
won't remove it, but I think that it will be much better to use
shell-script wrapper around it, probably taking more resources.
Currently you have to stop nncp-toss-cycle when you work with nncp-xfer,
to be able to run nncp-ack (because tossing will remove the packet)
before tossing. Moving ACKing to tosser -- will be a huge complication.
Currently I am in position that nncp-toss-cycle is not friendly with
nncp-xfer/nncp-bundle. It is even harmful, because while you copy files
from the external media, tosser will do tossing in background, requiring
to do read operations (at least) while nncp-xfer/bundle does writing at
the same time. So do not want to think about how all of that will work
with nncp-toss-cycle :-)

So currently I see two relatively simple changes I have to done:

* nncp-ack must (partly) decrypt packets, to determine if it ACK and skip it
* nncp-ack must generate list of generated ACK encrypted packets, to use
  it at least with nncp-rm, to be able to remove them after xfer transfer.
* nncp-rm could take a list of PKTs from stdin for convenience there
* probably nncp-xfer can take a list of packets (from the file/stdin) to
  skip. To be able to leave ACK packets in the spool just for a while

nncp-ack won't generate ACK for ACK, fixing the biggest problem.

-- 
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263  6422 AE1A 8109 E498 57EF