"are you bringing in user input from a web form into your shell scripts?"
This is shell scripting. We have literally decades of experience with these. We know for a positive fact that simple concatenation is dangerous. The contents of files, the names of files on the file system, and all the other things that shell scripts normally encounter are perfectly sufficient to wreck your day if you are too casual about them. Should you reply with something incredulous about this claim, be prepared for dozens of people to jump in with their war stories about how a single space in a filename in an unexpected place cost their business millions of dollars because of their shell script screwup. (Obviously, they are not that consequential on average, but the outliers are pretty rough in this case.)
Shell scripting is frankly dangerous enough just with what is on your system already; actually hooking up user input to it is borderline insane. Shell scripting would have to level up quite a bit to only be something to be concerned about when "user input" was being fed into it.
Learning to do half-decent shell scripting in most shells consists about half of learning the correct way to do things like pass through arguments from a parent to a child script, because for backwards compatibility reasons, all the easy ways are the wrong ways from the 1970s. It's nice when a more modern take on shell scripting is nicer than that.
I will also say when I'm evaluating libraries for things like shell scripting, I look for things like this, and it definitely doesn't score any points when I see stuff like this.
Just using `set -euo pipefail` will prevent many stupid things, but then again, conventional wisdom these days just seems to be to not use a shell if you can help it.
That's another example of what I mean, learning the magic invocations that amount to "Oh, please shell, act halfway like this is the 21st century, please?" It's like how I still have "#!/usr/bin/bash NL use strict; NL use warnings;" still burned into my fingers for Perl. (AIUI that's obsolete now but I never got to upgrade to the versions where that became obsolete, and now I'm just out of it.)
> We have literally decades of experience with these. We know for a positive fact that simple concatenation is dangerous.
Yes and no. I agree with the overall premise that the footguns are well documented, but at the same time, projects like this show that there are still large segments of developers who will gleefully shoot themselves in the foot because they never took the time to learn shell, or they just never had the opportunity to earn the battle scars.
The developers of a not-insignificant portion of IoT firmware absolutely are bringing user input in from web forms and chucking it into shell scripts. And unfortunately, it's that same class of developers that are disproportionately likely to pick up zx and run with it.
The point is, at this stage in the evolution of internet security, we pretty much know where the bugs come from. Injection attacks are still a huge practical problem. It would be nice if new scripting languages reduced that attack surface rather than increasing it.
In English, my daughter's given name has an optional apostrophe (Ma'ayan). I've seen systems that escape last-name apostrophes but not in the first name.
By not using eval (and sh -c, and everything equivalent) unless it’s absolutely unavoidable and always quoting variables. The $ construct acts basically like (the shell’s) eval. Proper escaping is an absolute must here.
I'm pretty sure that "eval is a bad idea" was mentioned in the "thought terminating cliches" topic on Ask HN a few days ago :)
In bash, `eval` is a footgun like any other. You can use it, but you just need to be aware of where your toes are when it shoots.
It's usually a "bad idea" in the sense that if you think you need to use it, you probably don't, and 90% of the time, there is an easier way to accomplish what you want to do. The next 5% of the time, using `eval` might be easier but will also create maintenance debt with its overgeneralization. And the final 5% of the time might be actual legit use cases for `eval`.
I just grepped my codebase for `eval` and I almost never use it. One example of the "overgeneralized" 5% might be when I realized I could use `eval` to set "variable variables" (i.e. the name of the variable is itself a variable, taken from a function argument). It was cool, but I ended up deleting it in favor of a more concrete solution.
Personally, if I'm hesitating to use `eval`, it's usually not for any security reasons. In general, my bash scripts only exist in dev machines and CI runners, and I don't copy them into the application containers that are exposed to a live runtime environment with untrusted users. So for CI/dev scripts, I can safely assume the code will only run in CI/dev, and therefore I can trust arbitrary user input (which I can of course still validate).
Thanks for the detailed response! I’ve been trying to level up my bash lately, and I definitely have seen a lot of ‘poor’ examples where they bailed out & use eval plus something else, for something native bash can in fact handle just fine, if you dig deep enough!
People always focus on the footguns that exist in Bash the language but ignore how much systems programming is abstracted away from you in the shell environment.
In Bash you can enter a Linux namespace with a single nsenter invocation. If you want to do the same in Python you have use ctypes and call libc.clone manually.
How is that a remotely real comparison? The ansible mv function deals with preserving SELinux context, which mv doesn't do, and it automatically deals with a whole lot of common error conditions, whereas mv just fails. If you just want to replicate mv, Python has shutil.move, one line of code. Ansible is trying to do a lot more.
- it's portable accross OSes (and keep flags if the OS supports it, deal with encoding, etc)
- it ensures selinux context is saved if there is such a thing
- it has proper and rich error communication with the calling code
- it's includes documentation and comments
- it outputs json par parsing and storage
No to say "mv" is not awesome, because it is. There is much more boiler plate in python, and is why I'll often do subprocess.check_call(['mv', 'src', 'dst']) if my script is linux only.
i think its not fair comparison.
mv command implementation in 'C' might have more lines of code. Maybe we should complain that there are no OOTB library functions in python to move the file.
I don't really care how many lines of code there are in an implementation. I care how many lines of code I actually have to write.
Python has shutil.move and os.rename but the Ansible example is to illustrate that there's a lot of code that needs to surround those calls to make them useful and they're not 1-1.
It also means that accidentally adding a space somewhere (“$HOME/go” -> “$HOME /go”) can have catastrophic effect. I wouldn’t dare write a single “rm” if I’m not 100% sure the argument is being quoted.
