Re: shell wildcard expansion (un)safety

["David A. Wheeler" <dwheeler () dwheeler com>]

> On Nov 5, 2024, at 11:12 PM, Solar Designer <solar () openwall com> wrote:
>
> Alexander Hu, CC'ed here, sent a message titled "shell expansion bug" to
> the distros list and a few other distro security contacts and shell
> maintainers.  The message described known and correct behavior (not a
> bug), even if unexpected by some and risky. ...

> Since this issue and other related ones were known for decades,
> getopt(3) and getopt_long(3), which are used by many programs, will stop
> processing options upon seeing a plain "--" argument.

However, many programs do *not* use getopt or getopt_long to process arguments.
Many programs support "--", but "not* all do,so using "--" as the sole countermeasure
requires careful review of every command's documentation.

I urge always using "./" to prefix wildcards if the first character is a wildcard,
e.g., "./*.pdf", because this ALWAYS works.

> ... over the years we gained things like ...
>
> find . -mindepth 1 -maxdepth 1 -type f -print0 | xargs -0 grep text --

The "-print0" and "-0" options have been widely implemented, but
POSIX 2024 finally formally adds them. So I urge using them where they
make sense, as they counter embedded linefeed characters in filenames.

> Can the shells do anything to mitigate this?  I think not without
> breaking compatibility.  The only not-too-unreasonable change I can
> think of is wildcard expansion prefixing filenames with "./", maybe only
> those that start with "-" and maybe not when used with builtin "echo".

I think something like this is a good idea, hopefully it'd be an option that could eventually
be standardized. I think "./" should be prefixed if the first character is a wildcard,
so that the resulting filenames will be consistent.

A simpler approach would be to simply forbid creating filenames that
include control characters or begin with "-". If you're doing that, also consider
an option requiring UTF-8 for new filenames. The current
situation makes it unnecessarily hard to write secure programs.
I wouldn't call supporting such filenames a "security vulnerability" exactly,
but they make developing secure software harder, and nothing *requires*
that we (as an industry) support them. POSIX never guaranteed such
filenames were allowed, and even has an error code for bad filenames.

Long ago I wrong a really long essay about POSIX filename issues.
Some people here may find it interesting:
https://dwheeler.com/essays/fixing-unix-linux-filenames.html

--- David A. Wheeler