public inbox for nncp-devel@lists.cypherpunks.ru Atom feed
* hjson file format @ 2024-03-12 22:53 Hadmut Danisch 2024-03-13 6:31 ` Sergey Matveev 0 siblings, 1 reply; 6+ messages in thread From: Hadmut Danisch @ 2024-03-12 22:53 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 1413 bytes --] Hi, just a question and proposal: I was preparing to try nncp on several machines, but found, that the hjson file format for /etc/nncp.hjson cannot easily distributed with automatic installation tools like ansible or puppet, because * it contains both public and secret data, and thus cannot be distributed as one file without modification, * it cannot be generated (easily) on the machine with common tools (to compose public and secret data into a single file), since it is not compatible to json and virtually no tool supports hjson. I therefore would like to make two proposals: 1. Either split the secrect parts (the self-key) and the public parts (shared with others) into two distinct files, to allow easy distribution and update of nncp.hjson or 2. alternatively allow use of either YAML file format, because YAML is designed to be easy human-readible and easy human-editable, but in contrast to hjson widely used and very well supported by a huge number of tools and programming library. In my eyes, YAML would be a better choice, since it also fulfills nncp's requirements, but it is even easier to read and edit ( no braces needed), and there's lot of tools to compose. And, YAML is a common standard for configuration files. If not YAML, then regular JSON, because there's tools for it (like jq). with best regards Hadmut [-- Attachment #2: Type: text/html, Size: 2049 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format 2024-03-12 22:53 hjson file format Hadmut Danisch @ 2024-03-13 6:31 ` Sergey Matveev 2024-03-13 17:34 ` Hadmut Danisch 2024-03-13 17:39 ` Sergey Matveev 0 siblings, 2 replies; 6+ messages in thread From: Sergey Matveev @ 2024-03-13 6:31 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 5147 bytes --] Greetings! *** Hadmut Danisch [2024-03-13 00:53]: >2. alternatively allow use of either YAML file format, because YAML is >designed to be easy human-readible and easy human-editable Absolutely not. Really, honestly, did you try using it? I dealt with it for years in various projects and completely confident that it is the worst format in terms of friendliness to human. I saw numerous times how people (including me) just can not interpret and understand what they see on the screen: is it a list of dictionaries, or vice versa, and similar kind of questions arose. Sorry, but YAML is just insanely complex to read and interpret by human, because of his rich set of expressions. * well-known "problem with Norway" countries: - GB - IE - FR - DE - NO * just a version: python: 3.5.3 postgres: 9.3 * just a surname: first name: Christopher surname: Null * just a fear how all of that could be interpreted: server_config: port_mapping: # Expose only ssh and http to the public internet. - 22:22 - 80:80 - 443:443 serve: - /robots.txt - /favicon.ico - *.html - *.png - !.git # Do not expose our Git repository to the entire world. flush_cache: on: [push, memory_pressure] priority: background * here is bet game: ipv4-address: 192.0.2.16 ipv6-address: 2001:db8::210 I used YAML in NNCP and several our of my personal projects, because of its ability to use templates and DRY your configuration file. But no, it is unusable because of those complexities with rich interpretation and abilities to expression yourself. I met too often how I and other people just were not able to tell what is encoded in YAML. In JSON you clearly see that "{}" gives your a dictionary, "[]" a list, '""' is about strings, nothing more. I think that YAML library's source code size was bigger than the whole NNCP project. And not all libraries support all set of features (equally) -- I saw that some of them did not support templates/references and one of the YAML's killer-feature is lost. And there are many revisions of YAML format itself. https://yaml.org/spec/1.2.2/ -- sorry, but that is just pure insanity to call that amount of text needed to be read by human as friendly to him. And without reading all of that: I will shoot myself in the foot. I also played with TOML format in NNCP. Do not remember did I leave the commit with it in the source code tree. It looked much more easier and simpler, but: [[foo.bar]] baz = 123 [[foo.bar]] abc = 123 becomes: { "foo": { "bar": [ {"baz": 123 }, {"abc": 123 } ] } } that is not-trivial to interpret in my opinion. Moreover those "[[foo.XXX]]" can be distributed across the whole configuration file. It is easy to deal by machine, but slightly more complex structures would be impossible for humans to conveniently create. Simple structures/configs are ok in TOML, but deeply nested like in NNCP were just barely readable. Maybe that is just because we are used to see all those "{}", "[]" in languages like Python and in widespread JSON and it is matter of taste? With TOML -- maybe. But YAML is just a no-no beast of complexity. For safer YAML usage there is Strict YAML subset. Also you can just wrap everything in quotes, to assure that it will be interpreted like a string. But here you would make some kind of JSON out of YAML already. >And, YAML is a common standard for configuration files. Popularity is pretty bad measurement. Microsoft Windows and JavaScript can be treated like standard (de-facto, by default) OS and programming language. Even not talking about systemd and Docker. >If not YAML, then regular JSON, because there's tools for it (like jq). Exactly that is why I use Hjson: it is interoperable with all JSON-aware tools. Its libraries are designed to convert it to/from JSON and then using ordinary JSON tools. http://www.nncpgo.org/Configuration.html contains a note, that Hjson can be trivially converted to JSON. That is why NNCP includes third-party hjson-cli utility for that purpose. Moreover, JSON is literally a valid subset of Hjson, so you can use it directly without any conversion. Anyway, none of those formats are truly convenient to be processed by machine. Once I started to write integration tests for NNCP and even with JSON it is pretty cumbersome to write all those *jq commands to generate a JSON file with multiple nested structures. It is possible, without any doubts, but not so easy, doing it with *jq utilities in shell. That is why I added possibility to use directory layout for the whole configuration: http://www.nncpgo.org/Configuration-directory.html Basically, each dictionary becomes a directory, each key is a filename, its contents is the value. -- Sergey Matveev (http://www.stargrave.org/) OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format 2024-03-13 6:31 ` Sergey Matveev @ 2024-03-13 17:34 ` Hadmut Danisch 2024-03-13 17:45 ` Sergey Matveev 2024-03-13 17:39 ` Sergey Matveev 1 sibling, 1 reply; 6+ messages in thread From: Hadmut Danisch @ 2024-03-13 17:34 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 1869 bytes --] On 13.03.24 08:31, Sergey Matveev wrote: > Really, honestly, did you try using it? I'm intensively using YAML for more then a decade in plenty of applications. As I mentioned, I'm using puppet (which stores all data in YAML) and ansible (which stores just everything in YAML), furthermore tools like cloud-init or CoreOS, and heaps of own software, all using YAML. But that's not the point. Was just a proposal. > Exactly that is why I use Hjson: it is interoperable with all JSON-aware > tools. Its libraries are designed to convert it to/from JSON and then > using ordinary JSON tools.http://www.nncpgo.org/Configuration.html > contains a note, that Hjson can be trivially converted to JSON. That is > why NNCP includes third-party hjson-cli utility for that purpose. > Moreover, JSON is literally a valid subset of Hjson, so you can use it > directly without any conversion. > > Anyway, none of those formats are truly convenient to be processed by > machine. Once I started to write integration tests for NNCP and even > with JSON it is pretty cumbersome to write all those *jq commands to > generate a JSON file with multiple nested structures. It is possible, > without any doubts, but not so easy, doing it with *jq utilities in > shell. That is why I added possibility to use directory layout for the > whole configuration:http://www.nncpgo.org/Configuration-directory.html > Basically, each dictionary becomes a directory, each key is a filename, > its contents is the value. > Nice, but misses the question. Which still leaves the question open: What is the intended clean way (without workarounds) to distribute the list of neighbors and their public keys, while on the same time keeping the secrets local on the machine, if they are both in the same file, but there is no tool to selectively modify parts of the file. regards Hadmut [-- Attachment #2: Type: text/html, Size: 2809 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format 2024-03-13 17:34 ` Hadmut Danisch @ 2024-03-13 17:45 ` Sergey Matveev 0 siblings, 0 replies; 6+ messages in thread From: Sergey Matveev @ 2024-03-13 17:45 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 788 bytes --] *** Hadmut Danisch [2024-03-13 19:34]: >Which still leaves the question open: What is the intended clean way >(without workarounds) to distribute the list of neighbors and their public >keys, while on the same time keeping the secrets local on the machine, if >they are both in the same file, but there is no tool to selectively modify >parts of the file. Convert that file to directory with nncp-cfgdir and do whatever modifications you wish with ordinary subdirectories/files. Or convert Hjson to JSON and modify it with JSON-aware utilities, if they are appropriate for that task (personally I used only gojq and gjo and not sure they are convenient enough). -- Sergey Matveev (http://www.stargrave.org/) OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format 2024-03-13 6:31 ` Sergey Matveev 2024-03-13 17:34 ` Hadmut Danisch @ 2024-03-13 17:39 ` Sergey Matveev 2024-03-13 21:54 ` Emery Hemingway 1 sibling, 1 reply; 6+ messages in thread From: Sergey Matveev @ 2024-03-13 17:39 UTC (permalink / raw) To: nncp-devel [-- Attachment #1: Type: text/plain, Size: 497 bytes --] By the way, one of NNCP users send me a (personal) message: >NixOS supports merging of configuration values and files using nncp-cfgdir. The same technique might be useful for Ansible and Puppet. >https://nixos.wiki/wiki/NNCP >https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/programs/nncp.nix#L72 So nncp-cfgdir is already used by someone more actively than me :-) -- Sergey Matveev (http://www.stargrave.org/) OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: hjson file format 2024-03-13 17:39 ` Sergey Matveev @ 2024-03-13 21:54 ` Emery Hemingway 0 siblings, 0 replies; 6+ messages in thread From: Emery Hemingway @ 2024-03-13 21:54 UTC (permalink / raw) To: nncp-devel Yea, that was me not using my mail client correctly. Using nncp-cfgdir is tricky but quite powerful. Excerpts from Sergey Matveev's message of March 13, 2024 5:39 pm: > By the way, one of NNCP users send me a (personal) message: > >>NixOS supports merging of configuration values and files using nncp-cfgdir. The same technique might be useful for Ansible and Puppet. >>https://nixos.wiki/wiki/NNCP >>https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/programs/nncp.nix#L72 > > So nncp-cfgdir is already used by someone more actively than me :-) > > -- > Sergey Matveev (http://www.stargrave.org/) > OpenPGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-03-13 22:13 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-03-12 22:53 hjson file format Hadmut Danisch 2024-03-13 6:31 ` Sergey Matveev 2024-03-13 17:34 ` Hadmut Danisch 2024-03-13 17:45 ` Sergey Matveev 2024-03-13 17:39 ` Sergey Matveev 2024-03-13 21:54 ` Emery Hemingway