public inbox for nncp-devel@lists.cypherpunks.ru
Atom feed
* Updates on NNCP stuff
@ 2023-10-31 17:41 John Goerzen
2023-10-31 20:03 ` Yggdrasil v0.5 dependency update Sergey Matveev
2023-10-31 20:38 ` MTH implementation Sergey Matveev
0 siblings, 2 replies; 4+ messages in thread
From: John Goerzen @ 2023-10-31 17:41 UTC (permalink / raw)
To: nncp-devel
Hi everyone,
I thought I'd mention a few things I'm looking at...
First of all, I'm writing a Rust library to interact with NNCP packets.
The idea is to expose a high-level API that is separate from the details
of configuration and file manipulation, as a way to learn more about
NNCP packets and to possibly use them in other contexts. (Random
thoughts: embed in emails, run through other queue systems, send over
UUCP, etc.) I don't plan to re-implement NNCPgo, and particularly not
the network protocol. My hope is to make it complimentary, to allow
things to hop across even more types of systems end-to-end, and to let
me do some research and experimentation on new types of auto-routing.
So far, I have a working parser for NNCP packets, complete with header
signature verification, decryption, and zstd decompression.
The next step is to fully test the different packet types (since I am
using the type system to decode them into more-specific structures; ie,
the path field is decoded as appropriate for different packet types.)
I hope to get the source code and crate up in a week or so.
The major thing I'm still puzzled about is writing a compatible Markle
tree hash. There are various libraries for doing this, but it looks
like the one in NNCP is home-grown and I'm scratching my head at the
source code. This isn't strictly necessary to parse packets, but it IS
necessary to interoperate with NNCPgo and to handle Ack packets, both of
which are goals of mine.
Secondly, I owe Sergey a documentation patch for the packet type enum.
I'll get that going.
Third, Yggdrasil just announced a major upgrade to 0.5. Although keys
and IPs remain the same, the wire protocol has an incompatible change.
It also requires the newer hjson v5 go module.
I happen to maintain both Yggdrasil and NNCP in Debian, and because of
the coupling between them (both for Yggdrasil and hjson), they likely
have to be upgraded in tandem. Sergey, if you have newer Yggdrasil and
hjson on your radar, that would be fantastic. Hopefully it takes
minimal effort, but I have no idea how these things tend to work in Go.
Thanks!
- John
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Yggdrasil v0.5 dependency update
2023-10-31 17:41 Updates on NNCP stuff John Goerzen
@ 2023-10-31 20:03 ` Sergey Matveev
2023-11-02 23:23 ` John Goerzen
2023-10-31 20:38 ` MTH implementation Sergey Matveev
1 sibling, 1 reply; 4+ messages in thread
From: Sergey Matveev @ 2023-10-31 20:03 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 698 bytes --]
Greetings!
*** John Goerzen [2023-10-31 12:41]:
>Third, Yggdrasil just announced a major upgrade to 0.5. Although keys
>and IPs remain the same, the wire protocol has an incompatible change.
>It also requires the newer hjson v5 go module.
Just made a 8.10.0 release with Yggdrasil, hjson and other dependant
libraries update (except for gvisor, that again changed its API, so I
did not bother dealing with it). Check Yggdrasil workability with my
v0.5 node.
Unfortunately tarball's size increased by nearly a megabyte, mainly
because Yggdrasil has QUIC transport now.
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: MTH implementation
2023-10-31 17:41 Updates on NNCP stuff John Goerzen
2023-10-31 20:03 ` Yggdrasil v0.5 dependency update Sergey Matveev
@ 2023-10-31 20:38 ` Sergey Matveev
1 sibling, 0 replies; 4+ messages in thread
From: Sergey Matveev @ 2023-10-31 20:38 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 2076 bytes --]
Greetings!
*** John Goerzen [2023-10-31 12:41]:
>The major thing I'm still puzzled about is writing a compatible Markle
>tree hash. There are various libraries for doing this, but it looks
>like the one in NNCP is home-grown and I'm scratching my head at the
>source code.
I did not find any Merkle-tree library with the ability to "prepend"
some data to partly existing tree and that will "fold" leaves/nodes
that are "ready". So indeed wrote my own one.
Actually that is pretty simple thing, does not requiring any
cryptography knowledge or anything complex. http://www.nncpgo.org/MTH.html
Instead of just hashing the whole data, you split it on 128KiB pieces
and hash each piece independently. But intead of ordinary pure
BLAKE3-256 hash, you use its keyed variant (I assume any BLAKE3
implementation library provides that option) with key="NNCP MTH LEAF".
Then you just hash each pair of resulting hashes with another BLAKE3-256
with the key="NNCP MTH NODE". And repeat that node hashing procedure
until only the single hash stays.
If there is only single block of data, then it is duplicated (not a
problem in NNCP usage context, where encrypted packets are
pseudorandom), to create two leaves. If there are no blocks, then hash
of the zero string is used as a leaf (and duplicated too). That are just
corner cases.
nncp/src/mth.go has so-called MTHFat implementation, that is just a
trivial implementation that keeps all hashes in memory all the time. Can
be used as an example. Complications with MTH appear only if you want to
start "hashing" data not from the beginning, but from some offset,
generating the tree from somewhere in the middle, folding some of the
nodes to save the memory and then "prepending" missing data from the
beginning to it. This is the case where you resumed download of some
file, automatically hash its part and after download finish you just
read from the disk its missing beginning.
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Yggdrasil v0.5 dependency update
2023-10-31 20:03 ` Yggdrasil v0.5 dependency update Sergey Matveev
@ 2023-11-02 23:23 ` John Goerzen
0 siblings, 0 replies; 4+ messages in thread
From: John Goerzen @ 2023-11-02 23:23 UTC (permalink / raw)
To: Sergey Matveev; +Cc: nncp-devel
On Tue, Oct 31 2023, Sergey Matveev wrote:
> Greetings!
>
> *** John Goerzen [2023-10-31 12:41]:
>>Third, Yggdrasil just announced a major upgrade to 0.5. Although keys
>>and IPs remain the same, the wire protocol has an incompatible change.
>>It also requires the newer hjson v5 go module.
>
> Just made a 8.10.0 release with Yggdrasil, hjson and other dependant
> libraries update (except for gvisor, that again changed its API, so I
> did not bother dealing with it). Check Yggdrasil workability with my
> v0.5 node.
Thank you, Sergey! I have uploaded NNCP 8.10.0, Yggdrasil 0.5.1, and
updates of their dependencies to Debian. NNCP 8.10.0 should be
available for all Debian unstable platforms by Saturday, and I will get
it in bookworm-backports in a couple of weeks as well.
> Unfortunately tarball's size increased by nearly a megabyte, mainly
> because Yggdrasil has QUIC transport now.
Wow, that's a lot of code. But, I guess, understandable, implementing a
whole protocol in userspace like that. That should have some very nice
properties for systems that hop networks (mobile to wifi and such).
- John
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-11-02 23:26 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-10-31 17:41 Updates on NNCP stuff John Goerzen
2023-10-31 20:03 ` Yggdrasil v0.5 dependency update Sergey Matveev
2023-11-02 23:23 ` John Goerzen
2023-10-31 20:38 ` MTH implementation Sergey Matveev