I am happy to announce a new major release of GNU poke, version 3.0.
This release is the result of a year of development. A lot of things have changed and improved with respect to the 2.x series; we have fixed many bugs and added quite a lot of new exciting and useful features. See below for a description of many of them. >From now on, we intend to do not one but two major releases of poke every year. What is moving us to change this is the realization that users have to wait for too long to enjoy new features, which are continuously being added in a project this young and active. The tarball poke-3.0.tar.gz is now available at https://ftp.gnu.org/gnu/poke/poke-3.0.tar.gz. GNU poke (http://www.jemarch.net/poke) is an interactive, extensible editor for binary data. Not limited to editing basic entities such as bits and bytes, it provides a full-fledged procedural, interactive programming language designed to describe data structures and to operate on them. Thanks to the people who contributed with code and/or documentation to this release. In certain but no significant order they are: Mohammad-Reza Nabipoor Arsen Arsenović Luca Saiu Bruno Haible apache2 Indu Bhagat Agathe Porte Alfred M. Szmidt Daiki Ueno Darshit Shah Jan Seeger Sergio Durigan Junior ... and yours truly As always, thank you all! But wait, this time we also have special thanks: To Bruno Haible for his invaluable advise and his help in throughfully testing this new release in many different platforms and configurations. To the Sourceware overseers, Mark Wielaard, Arsen Arsenović, and Sam James for their help in setting up the buildbots we are using for CI at sourceware. What is new in this release: User interface updates ====================== - A screen pager has been added to the poke application. If enabled with the `.set pager yes' option, output will be paged one screenful at a time. - A tracer has been added to libpoke and the poke application. If enabled with the `.set tracer yes' option, subsequent loaded Poke types will be instrumentalized so calls to user-defined handlers are executed when certain events happen: + Every time a field gets mapped. + Every time a struct/union gets mapped. + Every time a field gets constructed. + Every time a struct/union gets constructed. + Every time an optional field is omitted when mapping or constructing. This is very useful when debugging and writing pickles. - A new command sdiff (for "structured diff") has been added to the poke application, that provides a way to generate patchable diffs of mapped structured Poke values. This command is an interface to the structured diffs provided by the new diff.pk pickle. - When no name is passed to the .mem command, an unique name for the memory IOS with the form *N* will be used automatically, where N is a positive integer. - auto-completion of 'attributes is now available in the poke application. - Constraint errors now contain details on the location (which field) where the constraint error happens, along with the particular expression that failed. - Inline assembler expressions and statements are now supported: ,---- | asm (TEMPLATE [: OUTPUTS [: INPUTS]]) | asm TYPE: (TEMPLATE [: INPUTS]) `---- where PVM assembly code can be executed as part of normal Poke programs. - Both `printf' and `format' now support printing values of type `any'. - Both `printf' and `format' now support printing integral values interpreted as floating-point values encoded in IEEE 754. Format tags %f, %g and %e are supported. This feature, along with the new ieee754.pk pickle, eases dealing with floating-point data in binary data. - Pre-conditional optional fields are added to complement the currently supported post-conditional optional fields. A pre-conditional optional field like: ,---- | if (CONDITION) | TYPE FNAME; `---- makes FNAME optional based on the evaluation of CONDITION. But the field itself is not mapped if the condition evaluates to false. - A new option `.set autoremap no' can be used in order to tell poke to not remap mapped values automatically. This greatly speeds up things, but assumes that the contents of the IO space are not updated out of the control of the user. See the manual for details. - The :to argument to the `extract' command is now optional, and defaults to the empty string. - ${XDG_CONFIG_HOME:-$HOME/.config} is now preferred to XDG_CONFIG_DIRS. Poke Language updates ===================== - Array and struct constructors are now primaries in the Poke syntax. This means that it is no longer necessary to enclose them between parenthesis in constructions like: ,---- | (Packet {}).field `---- and this is now accepted: ,---- | Packet {}.field `---- - Bit-concatenation is now supported in l-values. After executing this code: ,---- | var a = 0 as int<4>; | var b = 0 as uint<28>; | | a:::b = 0x12345678; `---- the value of `a' is 0x1N and the value of `b' is (uint<28>) 0x2345678. - Arrays can now be indented by size, by specifying an offset as an index. This is particularly useful for accessing structures such as string tables without having to explicitly iterate on the array's elements. - Union types can now be declared as "integral". The same features of integral structs are now available for unions: integration, deintegration, the ability of being used in contexts where an integer is expected, etc. - Support for "computed fields" has been added to struct and union types. Computed fields are accessed just like regular fields, but the semantics of referring to them and of assigning to them are specified by the user by the way of defining getter and setter methods. - This version introduces three new Poke attributes that work on values of type `any': ,---- | VAL'elem (N) | evaluates to the Nth element in VAL, as a value of type `any'. | | VAL'eoffset (N) | evaluates to the offset of the Nth element in VAL. | | VAL'esize (N) | evaluates to the size of the Nth element in VAL. | | VAL'ename (N) | attribute evaluates to the name of the Nth element in VAL. `---- - Two new operators have been introduced to facilitate operating Poke array as stacks in an efficient way: apush and apop. Since these operators change the size of the involved arrays, they are only allowed in unbounded arrays. - Poke programs can now hook in the IO subsystem by installing functions that will be invoked when certain operations on IO spaces are being performed: ,---- | ios_open_hook | Functions in this hook are invoked once a new IO space has been | opened. | | ios_set_hook | Functions in this hook are invoked once the current IO space | changes. | | ios_close_pre_hook | ios_close_hook | Functions in these hooks are invoked before and after an IO space is | closed, respectively. `---- - The 'length attribute is now valid in values of type `any'. - Poke declarations can now be annotated as `immutable'. It is not allowed to re-define immutable definitions. - A new compiler built-in `iolist' has been introduced, that returns an array with the IO space identifiers of currently open IOS. - We have changed the logic of the EXCOND operator ?!. It now evaluates to 1 (true) if the execution of the first operand raises the specified exception, and to 0 (false) otherwise. We profusedly apologize for the backwards incompatibility, but this is way better than the previous (reversed) logic. - The containing struct or union value can now be refered as SELF in the body of methods. SELF is of type `any'. - Integer literal suffixes (B, H, U, etc) are case-insensitive. But until now little-case `b' wasn't being recognized as such. Now `1B' is the same than `1b'. - Casting to union types now raise a compile-time error. - If no explicit message is specified in calls to `assert', a default one showing the source code of the failing condition is constructed and used instead. - An operator `remap' has been used in order to force a re-map of some mapped Poke value. - Signed integral types of one bit are not allowed. How could they be, in two's complement? - The built-in function get_time has been renamed to gettime, to follow the usual naming of the corresponding standard C function. Standard Poke Library updates ============================= - New standard functions: ,---- | eoffset (V, N) | Given a value of type `any' and a name, returns the offset of | the element having that name. | | openset (HANDLER, [FLAGS]) | Open an IO space and make it the current IO space. | | with_temp_ios ([HANDLER], [FLAGS], [DO], [ENDIAN]) | Execute some code with a temporary IO space. | | with_cur_ios (IOS, [DO], [ENDIAN]) | Execute some code on some given IO space. `---- libpoke updates =============== - New API function pk_struct_ref_set_field_value. - New API function pk_type_name. Pickles updates =============== - New pickles provided in the poke distribution: ,---- | diff.pk | Useful binary diffing utilities. In particular, it implements | the "structured diff" format as described in | https://binary-tools.net/bindiff.pdf. | | io.pk | Facilities to dump data to the terminal. | | pk-table.pk | Convenient facilities to Poke programs to print tabulated data. | | openpgp.pk | Pickle to poke at OpenPGP RFC 4880 data. | | sframe.pk | sframe-dump.pk | Pickles for the SFrame unwinding format, and related dump | utilities. | | search.pk | Utility for searching data in IO spaces that conform to some | given Poke type. | | riscv.pk | Pickle to poke at instructions encoded in the RISC-V instruction | set (RV32I). It also provides methods to generate assembly | language. | | coff.pk | coff-aarch64.pk | coff-i386.pk | COFF object files. | | pe.pk | pe-amd64.pk | pe-arm.pk | pe-arm64.pk | pe-debug.pk | pe-i386.pk | pe-ia64.pk | pe-m32r.pk | pe-mips.pk | pe-ppc.pk | pe-riscv.pk | pe-sh3.pk | PE/COFF object files. | | pcap.pk | Capture file format. | | uuid.pk | Universally Unique Identifier (UUID) as defined by RFC4122. | | redoxfs.pk | RedoxFS files ystem of Redox OS. | | ieee754.pk | IEEE Standard for Floating-Point Arithmetic. `---- - The ELF pickle now provides functions implementing ELF hashing. Build system updates ==================== - It is now supported to configure the poke sources with --disable-hserver. Documentation updates ===================== - Documentation for the `format' language construction has been added to the poke manual. Other updates ============= - A new program poked, for "poke daemon", has been contributed to the poke distribution by Mohammad-Reza Nabipoor. poked links with libpoke and uses Unix sockets to act as a broker to communicate with an instance of a Poke incremental compiler. This is already used by several user interfaces to poke. - The machine-interface subsystem has been removed from poke, in favor of the poked approach. - The example GUI that was intended to be a test tool for the machine interface has been removed from the poke distribution. - Many bugs have been fixed. -- Jose E. Marchesi Frankfurt am Main 26 January 2023
