Building Executable Scripts For The Mac OSX Command Line
I'm relatively new to the Mac OSX operating system. And, as far as getting around on the command-line, I'm even more of a novice. I can't remember where I saw this, but recently, I saw a fellow programmer execute a set of command-line scripts from a file. What was so interesting to me about this, however, was that the file was invoked like a standard executable file. I think this is what a BAT file does on Windows. And in the Mac OSX / *NIX world, I think this is referred to as "Bash Scripting." Whatever it is, it seems really cool and I want to learn how to take advantage of it.
A couple of weeks ago, I wrote a small but non-trivial drag-n-drop JavaScript application in an effort to learn more about RequireJS, modular JavaScript application development, and MVC (Model-View-Controller) programming. As part of the exercise, I used the RequireJS compiler - r.js - to concatenate and minify all of the JavaScript dependencies. I ran the r.js compiler from the command line, providing it with the proper execution directory and a path to the build file. Rather than remembering how to run this compiler, however, I wanted to see if I could use a Bash script to simplify the execution.
Rather than executing Node.js from within the "js" directory, I wrote the following "executable" script in the project root:
## Use the "hash-bang" to tell the command-line environment which
## context to use when executing the following file. In this case,
## we are using BASH. I **believe** this line is optional in our
## case since BASH is the default environment.
#! /bin/bash
## Move execution to the JS directory as this is where we need to
## be in order for the "paths" in the build file to make sense.
cd ./js/
## Invoke the R.js optimizer using the build file.
node ../../r.js -o app.build.js
The first command line of the file is the "hash bang." This tells the command-line environment which context to use when executing the code. In this case, I am telling it to use the Bash bin; however, I believe that this line is optional (in this case specifically) since Bash is the default context (maybe) for the terminal.
Since we are in the root of the project, the second command tells the execution to move to the "js" directory where our JavaScript build file is. Then, once in the "js" directory, I use my installed Node.js runtime to invoke the "r.js" compiler along with the given build file.
By default, the text file I created - "build" - is not executable. As such, I had to change the permissions on it to include Execute:
chmod 777 ./build
The bit-mask of "777" is probably over-doing it; but, I don't know enough about file permissions to care about fine-tuning the setting.
Once the file was in place and the permissions had been applied, I was able to invoke the R.js compiler using the build file:
./build
Notice that I have to prefix the Bash file with "./". If I don't do that, I get the response:
-bash: build: command not found
Writing this simple command-line script created an "executable" that made compiling my RequireJS project incredibly easy! This Bash scripting looks powerful and will likely yield a lot of benefits when I can learn more about it. I'm sure this is old news to many of you out there, especially you Unix people; but for me, this is some awesome new hotness!
Want to use code from this post? Check out the license.
Reader Comments
Yeah bash scripts are nice. Still learning myself. Great for things like automating export/imports for MongoDB and the like.
The bang line is optional if you use the appropriate command from the command line when executing it,
for Bash. So, in either case, instead of using
you can use
to execute the Bash script.
File permissions: owner, group, everyone. Read access = 4, Write = 2, Execute = 1 (They add up to 7). So to give owner, group, and everyone else all permissions, that's 777. Usually regular files are 644, I think (O=WR, G=R, E=R). Leaving everything else the same, if you just want to make something executable (for everyone), just run `chmod +x [file]`.
To run you need to prefix the script with "./" otherwise the shell will search in your path for a file with this name. With the ./ you tell it to look in the current directory.
Permissions are easy: you have 3 numbers and each corresponds to a permission.
The first number is the owner permission, what you are allowed to do. The second number is the group permission, as each file is marked not just with who owns it, but which group owns it. The third is what everyone else is allowed to do.
Each number is a very simple 3-bit mask. From left (most significant) to right (least significant) you have r, w, x, or Read, Write, Execute.
So what does 777 mean? Each 7 in binary is 111, which means a combination of Read, Write, and Execute, for all 3 permissions. In other words, anyone can do anything they want with the file, including change it!
Also, this is why permissions print out the way they do when you do a directory listing: rwxrwxrwx. That's a bitmask, where each letter is a one and each dash is a zero.
Generally, for shell scripts you probably want to go with 744: you have full control, but others only have read access. If you're paranoid you could go with 700, so that you have full control but no one else can do absolutely anything with it.
The "chmod" utility makes this easy, too. If you want to set a file so that it's only executable for you, instead of remembering numbers and bitmasks you can say "chmod u+x whatever". That means "give the User Execute access".
@Jason C,
So, it's basically like Bash creating another bash process to execute the script? In the same way that running file through Node.js (hash-bang or bin file), would open up a Node process to execute the file?
@Jason,
Oh cool, I didn't know about the "+x" option. Thanks!
@Guillaume,
Ok, I just assumed all references would look in the current directory first, before then traversing the pathing options. Good to know! When I program (other languages), "current directory" is usually the default.
@Rick, @Jason,
Thanks for the insight into the 777 stuff. I used to know it at one point back in college, for when I took CompSci... but that was ages ago and has long since been forgotten.
I'll go with 744 for now - no one else uses this computer except me.
"So, it's basically like Bash creating another bash process to execute the script? [...]"
I'm not sure if that's how Node.js will work that's how it works for bash. It will fork itself into executing the script.
@Guillaume,
Sounds cool. I'm hoping to start leveraging this for fun and profit :D
@Ben,
You're right. bash is indeed the default shell for the most recent 2 or 3 versions of Mac OS X. Snow Leopard and Lion for sure. Not too sure about Leopard. Pretty sure Tiger and all preceding big cats defaulted to tcsh ("tee cee shell"). tcsh is very similar bash but Apple wanted to attract Linux users to Mac OS X, and bash is the default on Linux.
You should still specify shell at the start of scripts that you want to share with others, in case they use a different shell. I'm a C coder, so I still prefer the syntax of tcsh. The only time I ever use bash is when I need to redirect stderr, usually to throw it away (bash -c "my_script options 2>/dev/null", for example).
Happy scripting.
P.S.: That's how you specify shell outside the script. If I had specified it in the script, the bash -c would've been unnecessary.
... assuming the 2>/dev/null occurred in the script too that is. (Sheesh, mixing shells can get confusing.)
You should check out iTerm2 for your terminal needs - its better than the default Terminal the Mac comes with.
I've had to use a lot of unix utils for my job and they are definitely very powerful and fast. It is crazy how much raw data processing can be handled with the built-in unix tools.
This link has a bunch of useful utils that will come in handy at some point or the other:
http://www.quora.com/Linux/What-are-some-time-saving-tips-that-every-Linux-user-should-know
@Eapen,
You're right. About the only amazingly cool thing that Terminal has going for it is drag-and-drop of files and folders into the terminal window for command line completion.
Another cool terminal app for Mac is X11 from Apple (/Applications/Utilities/X11.app), which conveniently implicitly supports the X windowing system. It allows me to use the "Nirvana Editor" (command name "nedit", from nedit.org), which provides a very Mac-like set of keyboard combinations and syntax coloring. So nice to use Ctrl-C to mean Copy instead of Kill Foreground Process.
Hi Ben,
The `#!/bin/bash` is commonly referred to as the sha-bang (or she-bang or sh-bang), which is derived from sharp (#) and bang (!). I believe you'll need to make that the very first line of your script file, otherwise the command interpreter will just interpret (ignore) it as a comment. Then again, bash is probably your default command interpreter anyway, and it also doesn't look like you've used any bash-specific commands.
You do need to be careful about path/current directory in your scripts. Your example script works only if you call it as ./build, with your terminal working directory at the location of your script. Let's say you've located the script at /opt/bash/build, and you call it from a different location--say, /Users/Ben/--with the command `/opt/bash/build`. This will execute the script fine, but the initial command of `cd ./js/` will change the working directory to /Users/Ben/js/, not /opt/bash/js/. So, you might instead set a variable to your base directory, by leveraging the dirname command and the $0 variable, which holds the path to the script being executed:
So, your cd command would then become:
Or, you might just use this one-liner, instead of changing the working directory:
I'm far from a shell scripting guru, but I've really enjoyed the bash scripting I've gotten into, and have been able to make frequent use of bash scripts (and the very many very crazy powerful UNIX programs, like sed!). Here are some good links for you, whenever you feel like getting your bash geek on some more ;-)
http://tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html
http://tldp.org/LDP/abs/html/
http://www.gnu.org/software/bash/manual/bashref.html
Cheers!
Jamie
@WebManWalking,
Ha ha, thanks - trying not to get more confused ;)
@Eapen,
Thanks for the link. I'll check it out - I think there's a world of functionality here that I know nothing about!
@Jamie,
Thanks a lot for the links. Just from your code snippets, I'm seeing there's a lot of stuff here left to learn. For example, I assume the ${} is a way to insert variables into a string, ala string interpolation (or whatever that is called).
As to the "cd" command, I even felt uncomfortable writing that in the first place. I knew that I would always have to build the script from the root directory in order for it to make sense. The key will be to learn more about how this all works.
I just want to pipe something into something else! :D
@Ben,
LOL @ "I just want to pipe something into something else! :D"
I know, piping is still just so cool, and useful everywhere. Have a look at the sed command--it's a crazy powerful stream editor, and you can pipe stuff to sed all day long ;-)
http://www.grymoire.com/Unix/Sed.html
As for the ${}, the curly braces aren't always needed to output a variable in Bash, but they're helpful within certain strings.
Consider this...
So, simple variable output in a string is fine, as long as the variable name is not followed by a character that would be a valid part of the variable name. The curly braces are slightly more verbose, but can be a safe habit.
Great post, Ben. Coincidentally, I'm currently working on a set of "Beginning CentOS" posts for my blog that are along the same lines -- semi-introductory tasks on a Linux platform, for someone new to the OS...like me. :)
Repeat after me, UNIX is wonderful :-)
@Tom, UNIX is wonderful :D
@All,
I just tried rocking out this same concept using Node.js as the interpreter. Pretty awesome!
www.bennadel.com/blog/2329-Building-Executable-Scripts-For-The-Mac-OSX-Command-Line-With-Node-js.htm
I love leveraging skills I already have to do new things!
@Jamie, I even piped something!!!!
Better be careful Ben. Pretty soon you will changing your config files with sed, parsing log files with awk, grepping all over the place and running cron jobs instead of use CF scheduled tasks. But you should really worry when you start writing your code in vi.
I use bash scripts to start and stop CF, clean out parsed files etc. If I am forced to use a Windows machine the first thing I do is install Cygwin so I have access to *nix like command line.