public inbox for nncp-devel@lists.cypherpunks.ru
Atom feed
* hjson file format
@ 2024-03-12 22:53 Hadmut Danisch
2024-03-13 6:31 ` Sergey Matveev
0 siblings, 1 reply; 6+ messages in thread
From: Hadmut Danisch @ 2024-03-12 22:53 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]
Hi,
just a question and proposal:
I was preparing to try nncp on several machines, but found, that the
hjson file format for /etc/nncp.hjson cannot easily distributed with
automatic installation tools like ansible or puppet, because
* it contains both public and secret data, and thus cannot be
distributed as one file without modification,
* it cannot be generated (easily) on the machine with common tools (to
compose public and secret data into a single file), since it is not
compatible to json and virtually no tool supports hjson.
I therefore would like to make two proposals:
1. Either split the secrect parts (the self-key) and the public parts
(shared with others) into two distinct files, to allow easy
distribution and update of nncp.hjson or
2. alternatively allow use of either YAML file format, because YAML is
designed to be easy human-readible and easy human-editable, but in
contrast to hjson widely used and very well supported by a huge
number of tools and programming library. In my eyes, YAML would be a
better choice, since it also fulfills nncp's requirements, but it is
even easier to read and edit ( no braces needed), and there's lot of
tools to compose.
And, YAML is a common standard for configuration files.
If not YAML, then regular JSON, because there's tools for it (like jq).
with best regards
Hadmut
[-- Attachment #2: Type: text/html, Size: 2049 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format
2024-03-12 22:53 hjson file format Hadmut Danisch
@ 2024-03-13 6:31 ` Sergey Matveev
2024-03-13 17:34 ` Hadmut Danisch
2024-03-13 17:39 ` Sergey Matveev
0 siblings, 2 replies; 6+ messages in thread
From: Sergey Matveev @ 2024-03-13 6:31 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 5147 bytes --]
Greetings!
*** Hadmut Danisch [2024-03-13 00:53]:
>2. alternatively allow use of either YAML file format, because YAML is
>designed to be easy human-readible and easy human-editable
Absolutely not. Really, honestly, did you try using it? I dealt with it
for years in various projects and completely confident that it is the
worst format in terms of friendliness to human. I saw numerous times how
people (including me) just can not interpret and understand what they
see on the screen: is it a list of dictionaries, or vice versa, and
similar kind of questions arose. Sorry, but YAML is just insanely
complex to read and interpret by human, because of his rich set of
expressions.
* well-known "problem with Norway"
countries:
- GB
- IE
- FR
- DE
- NO
* just a version:
python: 3.5.3
postgres: 9.3
* just a surname:
first name: Christopher
surname: Null
* just a fear how all of that could be interpreted:
server_config:
port_mapping:
# Expose only ssh and http to the public internet.
- 22:22
- 80:80
- 443:443
serve:
- /robots.txt
- /favicon.ico
- *.html
- *.png
- !.git # Do not expose our Git repository to the entire world.
flush_cache:
on: [push, memory_pressure]
priority: background
* here is bet game:
ipv4-address: 192.0.2.16
ipv6-address: 2001:db8::210
I used YAML in NNCP and several our of my personal projects, because of
its ability to use templates and DRY your configuration file. But no, it
is unusable because of those complexities with rich interpretation and
abilities to expression yourself. I met too often how I and other people
just were not able to tell what is encoded in YAML. In JSON you clearly
see that "{}" gives your a dictionary, "[]" a list, '""' is about
strings, nothing more.
I think that YAML library's source code size was bigger than the whole
NNCP project. And not all libraries support all set of features
(equally) -- I saw that some of them did not support templates/references
and one of the YAML's killer-feature is lost. And there are many
revisions of YAML format itself. https://yaml.org/spec/1.2.2/ -- sorry,
but that is just pure insanity to call that amount of text needed to be
read by human as friendly to him. And without reading all of that: I
will shoot myself in the foot.
I also played with TOML format in NNCP. Do not remember did I leave the
commit with it in the source code tree. It looked much more easier and
simpler, but:
[[foo.bar]]
baz = 123
[[foo.bar]]
abc = 123
becomes:
{
"foo": {
"bar": [
{"baz": 123 },
{"abc": 123 }
]
}
}
that is not-trivial to interpret in my opinion. Moreover those
"[[foo.XXX]]" can be distributed across the whole configuration file. It
is easy to deal by machine, but slightly more complex structures would
be impossible for humans to conveniently create. Simple
structures/configs are ok in TOML, but deeply nested like in NNCP were
just barely readable.
Maybe that is just because we are used to see all those "{}", "[]" in
languages like Python and in widespread JSON and it is matter of taste?
With TOML -- maybe. But YAML is just a no-no beast of complexity.
For safer YAML usage there is Strict YAML subset. Also you can just wrap
everything in quotes, to assure that it will be interpreted like a
string. But here you would make some kind of JSON out of YAML already.
>And, YAML is a common standard for configuration files.
Popularity is pretty bad measurement. Microsoft Windows and JavaScript
can be treated like standard (de-facto, by default) OS and programming
language. Even not talking about systemd and Docker.
>If not YAML, then regular JSON, because there's tools for it (like jq).
Exactly that is why I use Hjson: it is interoperable with all JSON-aware
tools. Its libraries are designed to convert it to/from JSON and then
using ordinary JSON tools. http://www.nncpgo.org/Configuration.html
contains a note, that Hjson can be trivially converted to JSON. That is
why NNCP includes third-party hjson-cli utility for that purpose.
Moreover, JSON is literally a valid subset of Hjson, so you can use it
directly without any conversion.
Anyway, none of those formats are truly convenient to be processed by
machine. Once I started to write integration tests for NNCP and even
with JSON it is pretty cumbersome to write all those *jq commands to
generate a JSON file with multiple nested structures. It is possible,
without any doubts, but not so easy, doing it with *jq utilities in
shell. That is why I added possibility to use directory layout for the
whole configuration: http://www.nncpgo.org/Configuration-directory.html
Basically, each dictionary becomes a directory, each key is a filename,
its contents is the value.
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format
2024-03-13 6:31 ` Sergey Matveev
@ 2024-03-13 17:34 ` Hadmut Danisch
2024-03-13 17:45 ` Sergey Matveev
2024-03-13 17:39 ` Sergey Matveev
1 sibling, 1 reply; 6+ messages in thread
From: Hadmut Danisch @ 2024-03-13 17:34 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]
On 13.03.24 08:31, Sergey Matveev wrote:
> Really, honestly, did you try using it?
I'm intensively using YAML for more then a decade in plenty of
applications. As I mentioned, I'm using puppet (which stores all data
in YAML) and ansible (which stores just everything in YAML), furthermore
tools like cloud-init or CoreOS, and heaps of own software, all using YAML.
But that's not the point. Was just a proposal.
> Exactly that is why I use Hjson: it is interoperable with all JSON-aware
> tools. Its libraries are designed to convert it to/from JSON and then
> using ordinary JSON tools.http://www.nncpgo.org/Configuration.html
> contains a note, that Hjson can be trivially converted to JSON. That is
> why NNCP includes third-party hjson-cli utility for that purpose.
> Moreover, JSON is literally a valid subset of Hjson, so you can use it
> directly without any conversion.
>
> Anyway, none of those formats are truly convenient to be processed by
> machine. Once I started to write integration tests for NNCP and even
> with JSON it is pretty cumbersome to write all those *jq commands to
> generate a JSON file with multiple nested structures. It is possible,
> without any doubts, but not so easy, doing it with *jq utilities in
> shell. That is why I added possibility to use directory layout for the
> whole configuration:http://www.nncpgo.org/Configuration-directory.html
> Basically, each dictionary becomes a directory, each key is a filename,
> its contents is the value.
>
Nice, but misses the question.
Which still leaves the question open: What is the intended clean way
(without workarounds) to distribute the list of neighbors and their
public keys, while on the same time keeping the secrets local on the
machine, if they are both in the same file, but there is no tool to
selectively modify parts of the file.
regards
Hadmut
[-- Attachment #2: Type: text/html, Size: 2809 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format
2024-03-13 6:31 ` Sergey Matveev
2024-03-13 17:34 ` Hadmut Danisch
@ 2024-03-13 17:39 ` Sergey Matveev
2024-03-13 21:54 ` Emery Hemingway
1 sibling, 1 reply; 6+ messages in thread
From: Sergey Matveev @ 2024-03-13 17:39 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 497 bytes --]
By the way, one of NNCP users send me a (personal) message:
>NixOS supports merging of configuration values and files using nncp-cfgdir. The same technique might be useful for Ansible and Puppet.
>https://nixos.wiki/wiki/NNCP
>https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/programs/nncp.nix#L72
So nncp-cfgdir is already used by someone more actively than me :-)
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format
2024-03-13 17:34 ` Hadmut Danisch
@ 2024-03-13 17:45 ` Sergey Matveev
0 siblings, 0 replies; 6+ messages in thread
From: Sergey Matveev @ 2024-03-13 17:45 UTC (permalink / raw)
To: nncp-devel
[-- Attachment #1: Type: text/plain, Size: 788 bytes --]
*** Hadmut Danisch [2024-03-13 19:34]:
>Which still leaves the question open: What is the intended clean way
>(without workarounds) to distribute the list of neighbors and their public
>keys, while on the same time keeping the secrets local on the machine, if
>they are both in the same file, but there is no tool to selectively modify
>parts of the file.
Convert that file to directory with nncp-cfgdir and do whatever
modifications you wish with ordinary subdirectories/files. Or convert
Hjson to JSON and modify it with JSON-aware utilities, if they are
appropriate for that task (personally I used only gojq and gjo and not
sure they are convenient enough).
--
Sergey Matveev (http://www.stargrave.org/)
OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format
2024-03-13 17:39 ` Sergey Matveev
@ 2024-03-13 21:54 ` Emery Hemingway
0 siblings, 0 replies; 6+ messages in thread
From: Emery Hemingway @ 2024-03-13 21:54 UTC (permalink / raw)
To: nncp-devel
Yea, that was me not using my mail client correctly.
Using nncp-cfgdir is tricky but quite powerful.
Excerpts from Sergey Matveev's message of March 13, 2024 5:39 pm:
> By the way, one of NNCP users send me a (personal) message:
>
>>NixOS supports merging of configuration values and files using nncp-cfgdir. The same technique might be useful for Ansible and Puppet.
>>https://nixos.wiki/wiki/NNCP
>>https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/programs/nncp.nix#L72
>
> So nncp-cfgdir is already used by someone more actively than me :-)
>
> --
> Sergey Matveev (http://www.stargrave.org/)
> OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-03-13 22:13 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-12 22:53 hjson file format Hadmut Danisch
2024-03-13 6:31 ` Sergey Matveev
2024-03-13 17:34 ` Hadmut Danisch
2024-03-13 17:45 ` Sergey Matveev
2024-03-13 17:39 ` Sergey Matveev
2024-03-13 21:54 ` Emery Hemingway