Perl Search and Replace

Perl can be used to easily parse through files and perform a search and replace. For instance, the following command replaces all occurrences of ‘old’ with ‘new’ in myfile.txt after backing up the original as myfile.txt-OLD:

perl -pi-OLD -e 's/old/new/g' myfile.txt

Here is a description of the switches used according to Perl’s help (perl –help):

  • -p assume loop like -n but print line also, like sed
  • -i[extension] edit <> files in place (makes backup if extension supplied)
  • -e program one line of program (several -e’s allowed, omit programfile)

If you want to get creative, use Perl along with the Linux find command. With the next command, I replace ‘old’ with ‘new’ in all files that end in .html:

find /start/path -type f -name '*.html' -exec perl -pi-OLD -e 's/old/new/g' {} \;

Search and Replace Piped Output

You can also search and replace piped results. In this case, I display /etc/shadow and replace password hashes with ‘HIDDEN’:

sudo cat /etc/shadow | perl -p -e 's/(:)[^:]*/$1HIDDEN/'

Bash Math

Use Bash builtins (let, expr) for integer math and bc (a GNU numeric processing language) for floating point arithmetic in your bash scripts. Here are some examples.

Integer Math in Bash

The expr builtin can be used as a simple integer calculator. Results are rounded to the nearest integer and floating point is unknown. BE sure to escape the multiplication asterisks (*) to avoid Bash expansion.

bash$ expr 1 + 1
2
bash$ expr 3 \* 2
6
bash$ expr 6 / 3
2
bash$ expr 6 % 3
0
bash$ expr 3 / 2
1
bash$ expr 3 / 6
0
bash$ expr 6 \* 3.5
expr: non-numeric argument

Instead of expr(), you can also echo the output of $(( )) or $[ ].

bash$ expr 1 + 1
4
bash$ echo $((2 + 2))
4
bash$ echo $[2 + 2]
4

Use the bash builtin let for quick manipulation of bash variables.

bash$ NUM=41
bash$ let NUM+=1
bash$ echo $NUM
42

Floating Point Arithmetic in Bash

Using floating point in bash scripts requires an external calculator like GNU bc. Pipe your request to bc and note that escaping is not needed for quoted asterisks.

bash$ echo "3.8 + .4" | bc
4.2
bash$ echo '6 * 1.5' | bc
9.0

If all input values are integers, the bc option scale must be defined if you expect a floating point result.

bash$ echo '2 / 5' | bc
0
bash$ echo 'scale=2; 2 / 5' | bc
.40

You can also use the bash here string <<< to accomplish the same as a pipe of echo to bc:

bash$ echo 'scale=2; 2 / 5' | bc
.40
bash$ bc <<< 'scale=2; 2 / 5'
.40

Or, use bc -l to evoke the standard (but not default!) mathlib and see the result in floating point at max scale:

bash$ bc -l <<< '10.5 / 1'
10.50000000000000000000

Bash Random Numbers

Call the bash builtin variable $RANDOM to produce a random integer between 0 and 32767.

bash$ echo $RANDOM
194
bash$ echo $RANDOM
7601
bash$ echo $RANDOM
17576

For more control, use let to manipulate the number range. Examples below.

For a single digit Integer:

let R=$RANDOM%10; echo $R

For a number between 0 and 99

let R=$RANDOM%100; echo $R

For a number between 1 and 100

let R=$RANDOM%100+1; echo $R

Disable Auto Logout

Find that your shell is logging you out after a certain period of inactivity? You can tweak the number of seconds before the logout, or disable auto logout completely. In the instructions below, zero (0) disables auto logout or replace 0 with number for timeout seconds.

Disable Auto Logout in bash or sh

bash$ export TMOUT=0

Disable Auto Logout in csh or tcsh

tcsh% set autologout=0

Bash Colors

Bash Color Escape Codes

Echo (echo -e) the following escape codes inside \e[ESCCODEm to colorize text in Bash:

  • Black 0;30
  • Dark Gray 1;30
  • Blue 0;34
  • Light Blue 1;34
  • Green 0;32
  • Light Green 1;32
  • Cyan 0;36
  • Light Cyan 1;36
  • Red 0;31
  • Light Red 1;31
  • Purple 0;35
  • Light Purple 1;35
  • Brown 0;33
  • Yellow 1;33
  • Light Gray 0;37
  • White 1;37

Make sure to use echo -e to enable interpretation of backslash escapes:

bash$ echo -e "This is red->\e[00;31mRED\e[00m"

Remove Color

Echo \e[00m to remove text color modifications:

bash$ echo -n '\e[00m'

Bash Variable Variable

So, you want a Bash variable variable? You know, a variable that contains a variable name. No problem – Use indirect expansion or eval. Below are some examples and options.

Bash Indirect Expansion for Variable Variables

First, lets say you have a set of variables that contain integers.

bash$ ONE=1
bash$ TWO=2
bash$ THREE=3

Next, you want to store the variable name inside a variable called NUM.

bash$ NUM=TWO

This is how you expand $NUM to give you the value of the variable name it is storing.

bash$ echo ${!NUM}
2

Again, if you change $NUM to THREE:

bash$ NUM=THREE

You can expand $NUM to get the value of the variable name stored in $NUM:

bash$ echo ${!NUM}
3

Here is an example that provides you with the number of days in the current month.

#!/bin/bash
 
THISMONTH=$(date +'%b')
 
Jan=31 Mar=31 May=31 Aug=31 Oct=31 Dec=31
Apr=30 Jun=30 Sep=30 Nov=30
Feb="28 or 29"
 
echo "${!THISMONTH} days in $THISMONTH"
exit $?

Eval for Bash Variable Variables

You can alternatively use eval to accomplish variable variables:

bash$ FIVE=5
bash$ NUM=FIVE
bash$ eval echo \$$NUM
5

Here is an eval example that uses a variable as part of a variable name:

bash$ COLOR_RED=FF0000
bash$ COLOR_BLUE=0000FF
bash$ COLOR_GREEN=00FF00
 
bash$ MYCOLOR=RED
bash$ eval echo \$COLOR_$MYCOLOR
FF0000

Bash Socket Programming

You can connect to a socket using Bash by using exec and redirecting to and from the pseudo-path /dev/tcp/<hostname>/<port> or /dev/udp/<hostname>/<port>. For instance, to connect to your localhost SSH port using TCP:

exec 3<>/dev/tcp/localhost/22

Then, use cat and echo to read or write to the socket. Here is an example read:

cat <&3
SSH-2.0-OpenSSH_5.6

Notice that there is no such file as /dev/tcp or /dev/udp. Bash interprets the pseudo-path.

ls -l /dev/tcp
ls: cannot access /dev/tcp: No such file or directory
 
ls -l /dev/udp
ls: cannot access /dev/udp: No such file or directory

As another example, maybe you want to download a webpage:

exec 3<>/dev/tcp/www.fedora.org/80
echo -e "GET /\n" >&3
cat <&3

Finally, let's say you wanted to connect to an IRC server. Here is an example:

#!/bin/bash
 
##########################################################
# Config
 
NICK="mynick"
SERVER="irc.freenode.net"
PORT=6667
CHANNEL="#bashirc"
 
##########################################################
# Main
 
exec 3<>/dev/tcp/${SERVER}/${PORT}
echo "NICK ${NICK}" >&3
echo "USER ${NICK} 8 *  : ${NICK}" >&3
echo "JOIN ${CHANNEL}" >&3
cat <&3
 
exit $?

Sources

tldp.org: Advanced Bash-Scripting Guide - Chapter 29
thesmithfam.org: Bash socket programming with /dev/tcp