public inbox for nncp-devel@lists.cypherpunks.ru
Atom feed
From: Sergey Matveev <stargrave@stargrave•org>
To: nncp-devel@lists.cypherpunks.ru
Subject: Re: Avoiding double writes
Date: Mon, 1 Nov 2021 22:46:28 +0300 [thread overview]
Message-ID: <YYBEJ5DYaokI+bch@stargrave.org> (raw)
In-Reply-To: <87a6inkbsl.fsf@complete.org>
[-- Attachment #1: Type: text/plain, Size: 4074 bytes --]
Greetings!
*** John Goerzen [2021-11-01 12:47]:
>Awhile back, we were discussing the temporary files that were needed for
>reading stdin for nncp-exec or nncp-file. I believe the reason for this is
>that the header contains a signature of the data that follows, and it's not
>practical to seek back and write that later.
http://www.nncpgo.org/Encrypted.html
It is not because of signature, but mainly because of SIZE field.
Signed header contains everything you need to authenticate remote side
and create encryption keys to process ENCRYPTED data. There is not
signature over the whole packet. ENCRYPTED data contains encrypted size
as a first short block, and then pile of 128KiB blocks. Each block is
AEAD-encrypted, so it is authenticated. SIZE holds the payload size,
that can be smaller than the whole packet, because of added junk to hide
the actual payload size.
What can we do with the size, that anyway has to be known somehow.
Probably add its size as the very last block. But that way we have to
seek to the end (to read/decrypt it), then seek back to decrypt and copy
the actual payload. We can not do it in advance, because we do not know
where the actual payload stops and junk begins. Not an option.
We can store some structure inside each block (single signalling byte),
telling is it the payload one, or junk one. Now we can process it
sequentially. But junk in that case has to be some real data we will
really encrypt and authenticate. Junk can be zeroes, but AEAD-encryption
is much more expensive than current junk-generator made as BLAKE3-XOF
output. Much complication, more expensive (CPU). Ok, we know that if
junk block is started, then we can be sure that everything after will be
the junk, so we have to authenticate only the single block with the junk
and then quickly generate it without real AEAD-processing. Except for
the very last block with the size.
This is the option. That adds additional metadata to each block and
moves encrypted SIZE to the end. However we can not sequentially read
the packet and determine its size immediately: either we do seek, or
decrypt the data parsing it. So my main uncertainty is: is it worth of it?
Previously another stop-issue was the fact, that the whole generated
encrypted packet was hashed immediately, so we could not, for example,
leave fixed-sized SIZE block and fill it (seek back, then write) when
the whole data was read from stdin. Currently with MTH
(http://www.nncpgo.org/MTH.html) it can be done pretty efficiently: we
can hash only part of the data, and then hash another missing parts.
But NNCP allows encapsulating of transitional packets. So when I do:
nncp-file -via alice,bob - carol:dst, three encrypted packets are
generated on the fly feeding one to another: one is for carol, and
another two ones are transitional. So it complicated task of rewriting
SIZE field after the header more.
>To do that, there needs to be some way of recognizing the end of the data.
>I'm not sure how that happens now, since we also don't know the size in
>advance in those cases. Is there some sort of blocking for data chunks?
Storing of the whole data in temporary file is exactly intended for
gaining knowledge of the resulting size in advance. Plaintext is split
on blocks, which are independently AEAD-encrypted. 1) Most AEAD ciphers
interfaces allow only the whole data to be processed, without
intermediate updates, so we have to split it; 2) It allows exiting
decryption process immediately if one of the blocks already failed
(unauthenticated), without waiting till the whole packet was processed
and we saw invalid MAC. Initially NNCP used ordinary block cipher mode
with ordinary MAC function at the very end -- modern AEAD like
ChaCha20-Poly1305 is just faster and overhead of 16*8=128 bytes per
MiB of payload is negligible.
And of course probably I am just missing some damn simple solution.
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: CF60 E89A 5923 1E76 E263 6422 AE1A 8109 E498 57EF
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2021-11-01 19:46 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-01 17:47 Avoiding double writes John Goerzen
2021-11-01 19:46 ` Sergey Matveev [this message]
2021-11-02 0:11 ` John Goerzen
2021-11-02 10:03 ` Sergey Matveev
2021-11-02 15:26 ` John Goerzen
2021-11-02 17:49 ` Sergey Matveev
2021-11-02 20:48 ` Sergey Matveev