Bash File Loops

Some examples of iterating through files and looping through file content in Bash. For details, see the Bash man page or the Bash reference manual.

Loop Through Files

for FILE in *; do
  # do something with $FILE
  echo "File: $FILE"
done

For more control, use /usr/bin/file

for FILE in $(find ./ -name *.html -type f); do
  # do something with $FILE
  echo "HTML File: $FILE"
done

Loop Through Lines in a File

while read LINE; do
  # do something with $LINE
  echo "Line: $LINE"
done < /etc/hosts

Loop Through Words in a File

for WORD in $(cat /etc/hosts); do
  # do something with $WORD
  echo "Word: $WORD"
done

Loop Through Characters in a File

while read -n1 CHAR; do
  # do something with $CHAR
  echo "Character: $CHAR"
done < /etc/hosts

iTunes on Linux with Wine

iTunes for Windows XP can be run on Fedora 9 using Wine, an Open Source implementation of the Windows API on top of X and OpenGL. Please note that iTunes runs a bit slow over Wine and I have yet to test an iPhone or iPod with this configuration.

First, install all the wine packages.
sudo yum install wine*

Now, install the pulseaudio alsa plugin for sound support
sudo yum install alsa-plugins-pulseaudio*

Next, download the Windows XP iTunesSetup.exe from Apple.
firefox http://www.apple.com/itunes/download/

Now, install iTunes using wine
/usr/bin/wine iTunesSetup.exe

You can now run iTunes manually like so:
env WINEPREFIX="$HOME/.wine" wine "C:\Program Files\iTunes\iTunes.exe"

If you get an error regarding the registry, ignore it by clicking OK.

Here is a screenshot of iTunes running on Fedora 9.
Note that there is an icon for iTunes in Gnome's Notification Area (system tray).

For more information see the iTunes application page at winehq.org.

Top Linux IRC Clients

Internet Relay Chat (IRC), a form of Internet chat was created by Jarkko Oikarinen in 1988. Here is a list of Linux clients that will get you connected to IRC Networks. For an extensive list of IRC clients and their features, see the Wikipedia Comparison of Internet Relay Chat clients page.

  • Irssi
    Irssi is the self proclaimed client of the future and my favorite IRC client. Irssi is a terminal based, feature-rich, highly configurable, themed environment with support for Perl scripts. The project website hosts a large variety of themes and scripts that can be used with Irssi.

  • BitchX
    BitchX is a feature-rich and highly configurable, terminal based IRC client based on ircII. BitchX supports custom scripts, which can be written in TCL.
  • ChatZilla
    ChatZilla is a cross platform, graphical IRC client written entirely in JavaScript and XUL. ChatZilla is a mozilla.org project and can be installed as a Firefox plugin.

  • Konversation
    Konversation is a graphical IRC client for the KDE that supports multiple identities and theme support for nick icons.

  • Pidgin
    Pidgin is a Gnome based, multi-protocol Instant Messaging client with support for IRC.. With Pidgin, you can manage your instant messaging accounts and IRC connections with one, graphical client.

  • CenterIM
    CenterIM is a terminal based, multi-protocol Instant Messaging client with support for IRC. If you want the functionality of Pidgin in a text-based console, then CenterIM is for you.

Bash Trap Control C

Do you want to catch control-c keyboard interrupts in your Bash program? Use the Bash builtin trap command to catch system signals. The following runs control_c() when a user interrupts the main() section with a Control-C (SIGINT).

#!/bin/bash
 
cleanup()
# example cleanup function
{
  rm -f /tmp/tempfile
  return $?
}
 
control_c()
# run if user hits control-c
{
  echo -en "\n*** Ouch! Exiting ***\n"
  cleanup
  exit $?
}
 
# trap keyboard interrupt (control-c)
trap control_c SIGINT
 
# main() loop
while true; do read x; done

Bash & (Ampersand)

The Bash & (ampersand) is a builtin control operator used to fork processes. From the Bash man page, "If a command is terminated by the control operator &, the shell executes the command in the background in a subshell".

If logged into an interactive shell, the process is assigned a job number and the child PID is displayed. The job number below is one.

bash$ sleep 30 &
[1] 3586

Note that when a process is forked, the child PID is stored in the special variable $!

bash$ echo $!
3586

You can terminate the job by its job number like so:

bash$ jobs
[1]+ Running sleep 30 &
bash$ kill %1
[1]+ Terminated sleep 30
bash$

Bash Calculator

For floating point math calculations in Bash, use /usr/bin/bc.
A simple calculator program might look something like this:

#!/bin/bash
echo "scale=2; $@" | /usr/bin/bc -l
exit $?

Alternatively, a calculator function:

calc()
{
   echo "scale=2; $@" | /usr/bin/bc -l
   return $?
}

If you are looking for more, here are other ways to do math in Bash.

Clear and Disable Bash History

Need to clear your Bash history?
Use the Bash builtin history command:

history -c

To stop the writing of your Bash history to a file when you log out:

unset HISTFILE

See the Bash man page for details.

Ignore Python Deprecation Warnings

Need to silence or ignore Python Deprecation Warnings? Use:

#!/usr/bin/env python -W ignore::DeprecationWarning

Support One Laptop per Child

One Laptop per Child (OLPC) is at Linux World 2008 this week. Founded by Nicholas Negroponte, this non-profit organization is "dedicated to research to develop a low-cost, connected laptop, a technology that could revolutionize how we educate the world's children".

To participate in this great effort, visit: http://www.laptopfoundation.org/en/participate/


Prototype Ubuntu Mobile Device

At Linux World 2008, the Ubuntu booth had this prototype mobile device by Compal. The small, hand-held system was running Ubuntu MID Edition.

Compile Apache on Fedora

Below is a walk-through for compiling Apache 2.2 on Fedora 9. First, download and unpack the source tarball from Apache.org. As of August 2008, the latest version of Apache is 2.2.9 (httpd-2.2.9.tar.gz)

tar -xzvf httpd-2.2.9.tar.gz
cd httpd-2.2.9

Next, configure the source for build. You can provide configure with --help to see the large list of options. The example below enables mod_rewrite, mod_cache, mod_mem_cache and mod_ssl as shared modules in /opt/apache2/modules. Change --prefix to adjust your install PREFIX and adjust the configure options to suit your needs.

./configure --prefix=/opt/apache2 --enable-rewrite=shared --enable-cache=shared --enable-mem-cache=shared --enable-module=so

Now, build and install Apache.

make
sudo make install

Your Apache configuration will be in /opt/apache2/conf and you can control Apache with /opt/apache2/bin/apachectl. Refer to the Apache documentation for more details.

Linux World Free Exhibit Pass

If anyone wants to visit Linux World 2008 (San Francisco),
go to http://linuxworldexpo.com for a free exhibit hall pass.

Compile PHP on Fedora

Below is an outline on how to compile a feature rich, PHP Apache module on Fedora 9. First, compile Apache on the system using these instructions. This document assumes that Apache and the APache eXtenSion Tool are installed in the PREFIX: /opt/apache2.

Once you have Apache prepared, download the PHP source from php.net. As of August, 2008 the latest version is PHP 5.2.6. So, the following examples will use the php-5.2.6.tar.gz source tarball. Use your browser to download the latest and greatest. Then, untar the source.

tar -xzvf php-5.2.6.tar.gz
cd php-5.2.6

Next, configure the source for compile. Use the configure command. You can provide --help to see the large list of options.

./configure --help

For every option you enable, make sure you have the necessary RPMs installed to build PHP. For example, if you run configure with --with-snmp, you will need the net-snmp-devel RPM installed to complete the build. To ensure that you can build PHP with a full suite of options, install the following RPMs. Trim the list to suit your needs.

yum install aspell-devel curl-devel cyrus-sasl-devel e2fsprogs-devel freetype-devel glibc-devel keyutils-libs-devel krb5-devel libgcc libidn-devel libjpeg-devel libpng-devel libselinux-devel libsepol-devel libstdc++-devel libX11-devel libXau-devel libXdmcp-devel libxml2-devel libXpm-devel mysql-devel net-snmp-devel openldap-devel openssl-devel tcp_wrappers zlib-devel

Then, configure with the following settings. Change the configure --prefix to adjust your install PREFIX. Again, trim the list to suit your needs.

./configure --prefix=/opt/php5 --with-apxs2=/opt/apache2/apxs --with-mysql=/usr/bin/mysql_config --with-mysqli=/usr/bin/mysql_config --with-png-dir=/usr --with-gd --enable-gd-native-ttf --with-ttf --enable-safe-mode --enable-magic-quotes --with-pspell --with-gettext --with-jpeg-dir=/usr --with-zlib --with-curl --enable-soap --with-ldap=/usr --enable-sockets --with-openssl --with-snmp --enable-mbstring --with-freetype-dir=/usr --with-xpm-dir=/usr --with-libdir=lib64

Please note that if you are on a 32bit system, remove:

--with-libdir=lib64

Finally, build PHP and test it.

make
make test

If you are comfortable with the results (a few errors are OK in make test), then install PHP.

sudo make install

The install will place the PHP module in /opt/apache2/modules and prepare /opt/apache2/conf/httpd.conf with these settings:

LoadModule php5_module modules/libphp5.so

Then, place a php.ini in /opt/php5/lib (or your PREFIX/lib).

cp -a php.init-recommended /opt/php5/lib/php.ini

Finally, prepare your Apache Virtual Host to support index.php with the following code:

AddType application/x-httpd-php .php .inc .class
AddType application/x-httpd-php-source .phps
DirectoryIndex index.html index.php

Refer to PHP.net and Apache.org for reference.

Vi Search and Replace

Search and Replace

To begin a search and replace in Vi or Vim, start by hitting the colon <:>. This will allow you to enter a Vi command. Then, enter a search and replace command like so:

%s/old/new/g

This will search every line for 'old' and replace all occurrences in the line with 'new'. You start by mentioning which lines to affect. In my example, I use '%s'. This translates to all lines. Then, I trail the command with 'g' (for global) to affect all occurrences in the lines defined by '%s'. Note that global does not mean the entire file!

Limiting the Replace

You can declare the search and replace for only the current line by removing the percent sign:

s/old/new/g

It is also possible to limit the affected line numbers. The next example only affects lines 40 through 42:

40,42 s/old/new/g

Eliminating the 'g' from the end will only replace the first occurrence per line defined in the search. For example, the following replaces the first instance of 'hello' on every line in the file:

Regular Expressions

The search string can be the form of a regular expression. In the next example, I replace all numbers with the word 'number'.

%s/[0-9]/number/g

Escaping Characters

You will have to escape out the slash </> if it is part of your search string:

%s/http:\/\//https:\/\//g

Use Any Delimiter

Alternatively, change your delimiter. It can be anything!

%s!http://!https://!g

Get Confirmation Before Changing

You can get confirmation for each replace by adding a 'c' to the end of your command:

For more information, see the online Vi documentation.

Vi Search

In Vi or Vim, use the forward slash </> to search. Then type in your search string and hit <Enter>. You can navigate through occurrences of your search string using <n> to move forward and <N> to move backwards. Below is an example search for the string 'array'.

If you do not want Vim to highlight searches, set the following in your ~/.vimrc

set nohlsearch

To wrap searches around the file when using <n> and <N>, set the following in your ~/.vimrc

set wrapscan

For more, see this article if you need to search and replace.

Linux Wallpapers

Here are some of my favorite Linux Wallpapers (by various Flickr artists).



Bash Functions

Here are some example BASH functions. For more information, see the online BASH Function documentation or the BASH man page.

Declaring Functions

A BASH function must first be declared. One of the two methods can be used to declare a BASH function:

Method #1: function name compound-command
function say_goodbye {
  echo "goodbye"
}

Method #2: name() compound-command
say_hello() {
  echo "hello"
}

Using Functions

Just call the name as you would any other simple command:

say_hello
hello

Passing Variables

Pass variables into the function by space delimited fields and accept parameters using $1, $2 and so on.

my_function() {
  echo "$1"
}

Example:

my_function hello this is a test
hello

To expand all variable parameters inside the function, use $@

log() {
  echo "[$(date)]: $@"
}

log this is a test
[Sun Jun 22 10:21:26 PDT 2008]: this is a test

Return Codes

Functions return an exit status to their caller:

subtest() {
return 1
}

The function will return 1:

subtest || echo "Returned: $?"
Returned: 1

Since the function returned 1, it failed the test of || (or).

Free Software Sticker Book

Stickers Related to Free Software Projects (Mirror)

This book includes a set of stickers related to free software projects. Now, you may remove Microsoft sticker from your computer (computers are only not designed for the Microsoft Windows) and choose some sticker of this book to replace it. If you want to participate in Sticker Book Project, you can send your sticker/s to stickers@pikao.org.

Original Sites

Bash String Examples

Bash supports various string operations. Here are some examples to get you started quickly. For detailed information, see the official Bash documentation.

String Assignment

Below are a few ways to assign a string variable.

bash$ OS=Linux

Use quotes to encapsulate strings with spaces:

bash$ OS='CentOS 4.6'

Or, use double quotes to allow for variable expansion:

bash$ DISTRO="Fedora"
bash$ OS="$DISTRO Linux"
bash$ echo $OS
Fedora Linux

String Length

Determining the length of your string is easy:

bash$ PROG="Bash"
bash$ echo ${#PROG}
4

You can also use expr:

bash$ expr length $PROG
4

Substring Extraction

Below are some examples of substring expansion in the form of ${string:position} and ${string:position:length}. String indexing starts at zero!

bash$ PROG="Bash"
bash$ echo ${PROG:0}
Bash
bash$ echo ${PROG:1}
ash
bash$ echo ${PROG:1:2}
as

Testing Strings

Test if a string is of length 0

[ -z $STRING ]

Test if length of string is not zero:

[ ! -z $STRING ]
[ -n $STRING ]

Test if strings are equal.

[ $STRING1 == $STRING2 ]

Remember to use quotes if the string has spaces or escape characters (newlines).

[ "$STRING1" == "$STRING2" ]

The 'not equal; operator is !=

[ $STRING1 != $STRING2 ]

Fedora Apache Quickstart

If you are running Fedora and need Apache's HTTP web server, you have a couple of options. One, you can install Fedora's pre-built RPMs. Two, you can compile Apache to meet your specific needs.

Compiling Apache can improve performance and security. But, if you are starting out or need to get up and running quickly, use Fedora's RPMs.

Quick Install

To quickly install Fedora's Apache packages, install the 'Web Server' Yum group. This will install HTTPD with support for PHP, mod_perl, mod_ssl, and mod_python. The Web Server group also contains squid and weblizer.

yum groupinstall "Web Server"

If you want more control over the packages you need, install them one at a time. First, install HTTPD and then add on the modules you need.

yum install httpd
yum install mod_ssl mod_python

Apache Configuration

If you are feeling lazy and want to use a GUI, Fedora provides a graphical interface for configuring Apache. Install system-config-httpd:

yum install system-config-httpd

Then, run the tool as root:

/usr/bin/system-config-httpd

You can also configure the httpd.conf file manually. See the Apache documentation for details on how to configure your web server.

vim /etc/httpd/conf/httpd.conf
vim /etc/httpd/conf.d/README

Start Apache

You can set Apache to start at boot using chkconfig:

chkconfig httpd on

Finally, start Apache with the service command:

service httpd start

For more information, see Fedora's official documentation or the Fedora Documentation Project's Wiki.

Bash Arrays

Arrays in Bash are quite simple. Here are some examples to get you going. The official Bash documentation has more details and examples.

Initialize an entire array:

bash$ DAYS=(mon tue wed thu fri sat sun)

There are two ways to reference an entire array:

bash$ echo ${DAYS[@]}
mon tue wed thu fri sat sun
bash$ echo ${DAYS[*]}
mon tue wed thu fri sat sun

Initialize array element

bash$ ARRAY[0]="Fedora"
bash$ ARRAY[1]="RedHat"
bash$ ARRAY[2]="CentOS"

Display an element

bash$ echo ${ARRAY[0]}
Fedora

Get the length of an array:

bash$ echo ${#ARRAY[@]}
3

Get the length of an array value based on index:

bash$ echo ${ARRAY[0]}
Fedora
bash$ echo ${#ARRAY[0]}
6

Get the subset of an array through trailing substring extraction:

bash$ echo ${ARRAY[@]:0}
Fedora RedHat CentOS
bash$ echo ${ARRAY[@]:1}
RedHat CentOS
bash$ echo ${ARRAY[@]:2}
CentOS

Bash For Loop

Below are examples of the Bash for loop in action. If you need more help, see the official Bash documentation for an introduction to Bash programming.

Examples

Echo all jpg files in the current folder:

for JPG in *.jpg; do
  echo $JPG
done

Count a list of numbers:

for NUM in $(seq 1 10); do
  echo $NUM
done

One liners work as well:

for n in $(seq 3); do echo $n; done

Iterate through the process ID's for httpd and kill them:

for PID in $(ps -C httpd | awk '/httpd/ { print $1 }'); do
  kill -TERM $PID
done

List arguments to a function named showarg:

showarg() {
  for ARG in $@; do
    echo $ARG
  done
}

Loop through folders and back them up using tar:


DATE=$(date +'%Y-%m-%d')
for DIR in /etc /var /root; do
  tar -czvf /backups/$DIR_$DATE.tgz $DIR
done

Fedora 9 Perlcc

As of version 5.10.0-RC2, Perlcc is no longer distributed with Perl. This means that Fedora 9 does not include perlcc. Here is the official statement regarding perlcc from CPAN.org:

Removal of the bytecode compiler and of perlcc

perlcc, the byteloader and the supporting modules (B::C, B::CC, B::Bytecode, etc.) are no longer distributed with the perl sources. Those experimental tools have never worked reliably, and, due to the lack of volunteers to keep them in line with the perl interpreter developments, it was decided to remove them instead of shipping a broken version of those. The last version of those modules can be found with perl 5.9.4.

However the B compiler framework stays supported in the perl core, as with the more useful modules it has permitted (among others, B::Deparse and B::Concise).

Sources
CPAN.org: perldelta - what is new for perl 5.10.0

Vim Spell Check

Place these settings in ~/.vimrc to enable and configure spell checking in Vim. Comments start with double quotes. For more, open vim and type :help spell

Enable Spelling

set spell

Enable Spellfile

set spell spelllang=en_us
" zg to add word to word list
" zw to reverse
" zug to remove word from word list
" z= to get list of possibilities
set spellfile=~/.vim/spellfile.add

Set Colors

highlight clear SpellBad
highlight SpellBad term=standout ctermfg=1 term=underline cterm=underline
highlight clear SpellCap
highlight SpellCap term=underline cterm=underline
highlight clear SpellRare
highlight SpellRare term=underline cterm=underline
highlight clear SpellLocal
highlight SpellLocal term=underline cterm=underline

Fedora 9 Nvidia

Fedora 9 comes with two drivers that can be used in /etc/X11/xorg.conf to support your Nvidia card.

  1. NV: http://xorg.freedesktop.org/wiki/nv
  2. Nouveau: http://nouveau.freedesktop.org/wiki/

Although these two drivers work, they have their limitations. It is best to install the Nvidia driver from Nvidia.com. You can either build the source from Nvidia's Driver Website, or use Livna.org to install a binary kernel module.

Livna's Binary Nvidia Driver

To use Livna.org's binary driver, install Livna's release package for Fedora 9. This will provide you with access to Livna.org's Fedora 9 YUM repository using /etc/yum.repos.d/livna.repo.

rpm -ivh http://rpm.livna.org/livna-release-9.rpm

Next, install the Nvidia kernel module and all of it's dependencies:

sudo yum install kmod-nvidia

Then, use the nvidia-xconfig script to configure /etc/X11/xorg.conf. This will configure /etc/X11/xorg.conf with the 'nvidia' driver and it's options.

sudo /usr/sbin/nvidia-xconfig

Finally, reboot your system. You should see the Nvidia driver loading during boot. If you have any X errors, check /var/log/Xorg.log. If you want to tweak your driver, OpenGL or display settings, use nvidia-settings.

sudo /usr/bin/nvidia-settings

Script for Installing Flash on FC9

I have posted instructions on how to install Flash on Fedora 9, but Laurent Armstrong wrote me to share his script for automating the Flash install process -- and more. Laurent's script is interesting, so you might want to check it out: http://vegasoft.ca/howto/Install_f9

Thanks for the email, Laurent!

Update on Bash Random Numbers

Kuba wrote me from Poland, with some interesting ways to limit the scope of Bash's $RANDOM variable. I updated my instructions for generating random numbers in Bash to include Kuba's suggestions.

Thanks, Kuba!

Update on Fedora 9 GDM Backgrounds

Todd Zullinger, recently showed me how to change the Fedora 9 GDM background using a gconftool-2 one liner. So, I updated my Fedora 9 GDM instructions to include Todd's suggestion.

Thanks Todd!

Argument List Too Long

Getting an "Argument list too long" error using Linux?

This is because you are passing too many arguments to an executable.
For example, using *.html below results in too many arguments to /bin/rm:

rm *.html
bash: /bin/rm: Argument list too long

One solution is to use Bash control loops to iterate through arguments.

for HTMLFILE in *.html
> do
> rm -f $HTMLFILE
> done

Alternatively, the find command can be used to manipulate large data sets.
The following find command is identical in result to the for loop shown above.

bash$ find ./ -maxdepth 1 -name '*.html' -exec rm -f {} \;

Syndicate content