While exploring null bytes in this issue I learned that the hexdump command on macOS (and presumably other Unix systems) has a confusing default output.
Consider the following:
$ echo -n 'abc\0' | hexdump
0000000 6261 0063
0000004
Compared to:
$ echo -n 'a' | hexdump
0000000 0061
0000001
I'm using echo -n here to avoid adding an extra newline, which encodes as 0a.
My shell hell is zsh - bash requires different treatment, see below.
How come abc\0 starts with 6261 where a starts with 0061?
It turns out hexdump default format is 16-bit words in little-endian format, which is really confusing.
Using the -C option fixes this:
$ echo -n 'a' | hexdump -C
00000000 61 |a|
00000001
$ echo -n 'abc\0' | hexdump -C
00000000 61 62 63 00 |abc.|
00000004
C here stands for "canonical".
In addition to causing hexdump to output byte by byte, it also includes an ASCII representation on the right hand side.
Karl Pettersson pointed out that these examples won't work on Bash.
I ran bash on my Mac and found the following:
bash-3.2$ echo -n 'abc\0' | hexdump -C
00000000 61 62 63 5c 30 |abc\0|
00000005
bash-3.2$ echo -n $'abc\0' | hexdump -C
00000000 61 62 63 |abc|
00000003
bash-3.2$ printf 'abc\0' | hexdump -C
00000000 61 62 63 00 |abc.|
00000004
bash-3.2$ printf $'abc\0' | hexdump -C
00000000 61 62 63 |abc|
00000003
So it looks like using printf 'abc\0' is the best recipe for Bash on macOS. I'm not sure if Bash on other platforms differs.
Bill Mill suggested echo -ne for this:
bash-3.2$ echo -ne 'abc\0' | hexdump -C
00000000 61 62 63 00 |abc.|
00000004
The -e option enables the interpretation of backslash escapes.
Created 2023-05-22T11:01:10-07:00, updated 2023-05-22T12:17:57-07:00 · History · Edit