Using Perl LWP::UserAgent to View Server Headers

Unless configured to do otherwise, most HTTP and FTP servers will supply you with identifying information in the form of a server header. Using Perl’s LWP::UserAgent, you can connect to a server and display the header information.

First, you’ll need LWP::UserAgent. You can install this using Perl’s CPAN repositories:

sudo perl -MCPAN -e 'install LWP::UserAgent'

After LWP::UserAgent is installed, you can use Perl to connect to the server and download the information. Here is an example script which takes one argument and displays the information:

#!/bin/env perl
 
use warnings;           # keep us warned
use strict;             # keep us honest
use LWP::UserAgent;     # for web requests
 
my $url;                # init url var
my $serverheader;       # init server header var
 
# require one argument
if ($#ARGV != 0) {
 
    printf("Usage: %s \n", $0);
    exit(1);
 
} else { $url = $ARGV[0]; }
 
# build connection properties
my $ua = LWP::UserAgent->new();
$ua->timeout(10);
$ua->agent('Mozilla/5.0');
 
# connect and get
my $response = $ua->get($url);
 
# if we can connect...
if ($response->is_success) {
 
    # grab server header
    $serverheader = $response->server;
 
    # if server header exists...
    if (defined($serverheader)) {
 
        # print server header
        printf("%s\n", $response->server);
 
    # else, print a message
    } else { printf("No server header available.\n"); }
 
# else, print connection status
} else { printf("%s\n", $response->status_line); }
 
exit(0);

Assuming that the code is saved as getinfo.pl and executable (chmod +x), you can use it to fetch interesting information from various HTTP and FTP servers.

bash$ ./getinfo.pl http://www.apache.org
Apache/2.3.15-dev (Unix) mod_ssl/2.3.15-dev OpenSSL/1.0.0c

bash$ ./getinfo.pl ftp://apache.mirrors.pair.com
apache.mirrors.pair.com NcFTPd Server (licensed copy)

Of course, some sites do not publish HTTP headers (but most do).

bash$ ./getinfo.pl http://www.facebook.com
No server header available.

Finally, the code will show you if there is an error:

bash$ ./getinfo.pl http://badurl
500 Can't connect to badurl:80 (Bad hostname 'badurl')

Perl For Loop

The Perl for loop is used to loop through a block of code until a specified condition is met. The for loop statement contains three sections followed by a block of code. Below is an example.

A Simple For Loop Example


for (my $number = 1; $number <= 10; $number++) {
  print "$number ";
}

The first section initializes a variable


my $number = 1;

Then, the loop condition is provided. The loop will run as long as this is true.


$number <= 10;

Finally, the last section is executed at the end of each loop iteration. In our example's case, $number increments by one.


$number++

Executing this loop results in:


1 2 3 4 5 6 7 8 9 10

For Loop and Arrays

Here is an example of how the for statement can be used to loop through an array.


my @languages = ("Perl", "Python", "C", "Fortran");
my $size = @languages;
for (my $i = 0; $i <= $size; $i++) {
  print "$languages[$i]\n";
}

Fedora 9 Perlcc

As of version 5.10.0-RC2, Perlcc is no longer distributed with Perl. This means that Fedora 9 does not include perlcc. Here is the official statement regarding perlcc from CPAN.org:

Removal of the bytecode compiler and of perlcc

perlcc, the byteloader and the supporting modules (B::C, B::CC, B::Bytecode, etc.) are no longer distributed with the perl sources. Those experimental tools have never worked reliably, and, due to the lack of volunteers to keep them in line with the perl interpreter developments, it was decided to remove them instead of shipping a broken version of those. The last version of those modules can be found with perl 5.9.4.

However the B compiler framework stays supported in the perl core, as with the more useful modules it has permitted (among others, B::Deparse and B::Concise).

Sources
CPAN.org: perldelta – what is new for perl 5.10.0

Perl Search and Replace

Perl can be used to easily parse through files and perform a search and replace. For instance, the following command replaces all occurrences of ‘old’ with ‘new’ in myfile.txt after backing up the original as myfile.txt-OLD:

perl -pi-OLD -e 's/old/new/g' myfile.txt

Here is a description of the switches used according to Perl’s help (perl –help):

  • -p assume loop like -n but print line also, like sed
  • -i[extension] edit <> files in place (makes backup if extension supplied)
  • -e program one line of program (several -e’s allowed, omit programfile)

If you want to get creative, use Perl along with the Linux find command. With the next command, I replace ‘old’ with ‘new’ in all files that end in .html:

find /start/path -type f -name '*.html' -exec perl -pi-OLD -e 's/old/new/g' {} \;

Search and Replace Piped Output

You can also search and replace piped results. In this case, I display /etc/shadow and replace password hashes with ‘HIDDEN’:

sudo cat /etc/shadow | perl -p -e 's/(:)[^:]*/$1HIDDEN/'