lsof and Unclosed file handlers

After checking out a blog entry about analyzing Classpath of a running Java program by Thilina Mahesh Buddhika, i decided to try the command on the Java Web server used at my organization. To my surprise, i was able to find unclosed file handlers in the application hosted on the Web server.

The command lsof -p lists all open sockets, files and pipes. By analyzing the output of the command, i was able to check all the files opened by the application and able to reason why it was open by going through the application code. This also helped me in fixing unclosed file handlers.

Try out the command and try figuring out why the files are referred to. This gives an insight to functioning of the application.

Check out the usage of the command @ following link http://sial.org/howto/debug/unix/lsof/

remsh and ftp – Automate copying files between machines

Update on 26′th Aug 2009.

Just use rcp or scp

Ignore the rest of the blog article.

Consider a scenario where you need to get files from multiple remote machine to your machine and do a task like comparing or auditing, etc, the way it is done is is to do a remote login to a machine and do a ftp for copying a file from the remote machine to your machine

rlogin <remote-machine-name> -l <user>

ftp <my-machine-name>

....
...

and then doing the actual task. It is highly repetitive task to login to different machines and ftping the files. Also, as far as i know automating to do a remote login and ftping the file is impossible using shell script. Please let me know if there is a way to do it via scripts.

I have found a better way to do it using remsh command and using automated ftp script.

Consider the following scenario. There are two unix boxes. One, machine001 and another machine002. For machine machine001, you have direct access using m001username and m001password and to machine002 you do a remote login from machine001 using the user id m002username.

Suppose, you need to get a file file001 from machine002 to machine001.

We do it in following way.

machine001>rlogin machine002 -l m002username
blah blah messages
machine002>ftp machine001
>username m001username
>password ************
>put file001 dir001/file001
>bye
machine002>exit
machine001>ls dir001
file001

Doing the same using ftp and remsh is explained below. The automated ftp script is as follows. It accepts target machine name, username, password, the source file and destination file. This is a simple shell script whose functionality can be greatly extended.

automateFtp.sh

#!/bin/ksh
numOfArgs=$#
if [[ $numOfArgs = "5" ]]
then
        machineName=$1
        username=$2
        password=$3
        sourcefile=$4
        destFile=$5
#Start the ftp process to sent the file
        ftp -n -v ${machineName} <<-ENDTAG
        user ${username} ${password}
        put ${sourcefile} ${destFile}
        quit
ENDTAG
#End of ftp
else
        echo "Insufficient number of arguments"
fi

Just copy this file to the machine whose file needs be copied over to your machine and invoke the remote script using the remsh command.

In the above scenario, the command would be

remsh machine002 -l m002username automateFtp.sh machine001 m001username m001password
file001 dir001/file001

i.e., generally

remsh <from_machine> -l <from_machine_login_name> automateFtp.sh <to_machine>
<to_machine_username> <to_machine_password> <from_machine_file_name><to_machine_file_name>

Since the shell script is a simple one, it can be modified to try out in various scenarios.

This helps in automation to great extent. Just copy automateFtp.sh to each of the machines and invoke them from your machine. Now the task is just reduced to invoking a command. This can also be automated.
Since you pass the username and password at execution time, they are safe from prying eyes.

Unix Script + Java = Solve tasks easily.

The example given below shows the way the unix scripts and other programs (in this case, it is java program) can be combined to tackle simple problems.

The example script calls a java program to check the status of a URL. The Shell script gets the list of all URLs that needs to be checked via a file urlList and checks the status of each of the URL via the java program com.util.GetHttpResponse and then if the response is not equal to 200, sends a mail.

The java program to check the status of the URL can be replaced by any program that provides the same functionality. This program shows how unix shell scripts can be combined effortlessly to solve tasks.

There are a numerous way to get the status code of a HTTP URL. The post shows using Java. This simple tool can be used to check the status of a list of URLs and if down send information to the concerned person. This script can be invoked through cron to regularly check the state of URLs. This can be extended to check the state of other protocols also.

The unix shell script.

#!/bin/ksh
# Visit http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html for
# more information in HTTP Status Response Code
# Response of 200 specifies successful Request

for eachUrl in `cat urlList`
do
	responseStatusCode=`java com.util.GetHttpResponse $eachUrl`
	echo "URL: $eachUrl RESPONSE CODE: $responseStatusCode"
	if [[ $responseStatusCode != "200" ]]
	then
		mailx -s "The URL $eachUrl did not return successful
                          request status. The HTTP response status is
                          $responseStatusCode. Please Check."
                          someone@somewhere.com
	fi
done

The java Class com.util.GetHttpResponse.

package com.util;

import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;

public class GetHttpResponse {

	public static void main(String[] args) {
		new GetHttpResponse().getWebPage(args[0]);
	}

	public void getWebPage(String pageUrl) {
		if (pageUrl == null) {
			throw new IllegalArgumentException("pageUrl cannot be null");
		}
		try {
			URL webPageUrl = new URL(pageUrl);
			HttpURLConnection conn = (HttpURLConnection) webPageUrl.openConnection();
			conn.setRequestMethod("GET");
			System.out.println(conn.getResponseCode());
			conn.disconnect();
		} catch (MalformedURLException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}

For Complete details on HTTP Status response code, visit

http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html