Showing posts with label Solaris. Show all posts
Showing posts with label Solaris. Show all posts

Monday, October 27, 2008

Watchdog Script for a Process

Many long running processes like to just disappear at random intervals, especially late in the day on Friday so that a client calls you at 5pm to fix it. While the following script doesn't solve the root cause of why your program is dying, sometimes there is no reasonable way to fix the root cause and a watchdog timer script will do the job.

Watchdog Script

I've written this with the intention of restarting a Tomcat based service, but if you fiddle with the arguments to FIND_PROC, you can find whatever you want. Just ONE BIG caveat: don't name your watchdog script so that the watchdog finds itself.

The first version of this script works on Solaris:
#!/bin/sh

# Find the pid of the process (PPID will be the shell that started it
) remember no spaces allowed between varnames, just equals sign, and the value
FIND_PROC=`pgrep myprocess`
#FIND_PROC='pgrep testscript`
# if FIND_PROC is empty, the process has died; restart it

if [ -z "${FIND_PROC}" ]; then
echo myprocess failed at `date`
nohup /export/home/path/to/your/process.sh
#nohup /export/home/admin/testscript.sh
fi

exit 0


If you can't use pgrep, there's another way to do this. You can use grep and awk.
FIND_PROC=`ps -ef |grep myprocess | awk '{if ($8 !~ /grep/) print $2}'`

ps picks up the grep process, so the awk script removes this from the listing. The column numbers will likely need adjusting-- just change "print $2" to "print" and run the command from the command line and you will see the whole like. Determine which column holds the PID.

You can read more about awk here.

You might also need to make the ps listing wider than the default 80 characters to avoid having the process name cut off. In this case, your ps may support the --columns argument.

Cron Entry

To edit the cron file, you may first need to set the correct editor. I am assuming this is vi (Solaris likes you to use the absolutely atrocious "ed" by default-- even now in 2008... as if vi isn't archaic enough). If you get stuck inside ed, just type "quit".
export EDITOR="vi"
Then run the crontab program
crontab -e
It may very well be empty if you haven't set up an cron jobs. The format is a columns separated by spaces. Minute, hour, day of month, month, day of week, command to execute. You can either put a number for the time columns, a list of numbers (separated by commas), a range (using a dash) or an asterisk (*) to specify all.

I used the following line for my watchdog script (which logs normal and error output to watchdog.log):
0,10,20,30,40,50 * * * * /export/home/admin/watchdog.sh >> /export/home/admin/wa
tchdog.log 2>&1

(note, I couldn't paste the above line into crontab -e, but I could retype it no problem... go figure)

Wednesday, December 20, 2006

Starting Sun Appserver 8.1 without Typing in a Password

Since the Appserver tends to die quite often, this is good for adding a cronjob to cycle the thing every day.


Edit

/opt/SUNWappserver/sbin/passwordfile



Add the following:

AS_ADMIN_password=XXXXX



then start asadmin like so:

/opt/SUNWappserver/sbin/asadmin start-domain --user admin --passwordfile 
/opt/SUNWappserver/sbin/passwordfile domain1

(note that there should be no line wrap above)

Thursday, May 04, 2006

Quickly Add Swap on Solaris

In a crunch moment, you might realize, this server doesn't have enough memory and swap. What do you do? You don't have time to redo the filesystem or add another disk.

This link explains how to create a swap file on Solaris: http://www.eyrie.org/~eagle/notes/solaris/swap-file.html

The skinny of it is to run

/usr/sbin/mkfile SIZEinMBm /SWAP_FILE_NAME

Then add the swap file to the list of swap space:

/usr/sbin/swap -a /SWAP_FILE_NAME

Then run swap -l to make sure the swap space is being used.

Wednesday, April 19, 2006

Mounting an external USB disk in Solaris

1) Plug in the USB drive.

2) Run
dmesg
. Make sure the system found the drive.
Apr 19 11:49:35 accelere12 usba: [ID 855233 kern.info] USB-device: storage@3, scsa2usb0 at bus address 2
Apr 19 11:49:35 accelere12 usba: [ID 593373 kern.info] Cypress Semiconductor USB2.0 Storage Device DEF106F1A17F
Apr 19 11:49:35 accelere12 genunix: [ID 936769 kern.info] scsa2usb0 is /pci@8,700000/usb@5,3/storage@3
Apr 19 11:49:35 accelere12 genunix: [ID 408114 kern.info] /pci@8,700000/usb@5,3/storage@3 (scsa2usb0) online
Apr 19 11:49:37 accelere12 scsi: [ID 193665 kern.info] sd30 at scsa2usb0: target 0 lun 0
Apr 19 11:49:37 accelere12 genunix: [ID 936769 kern.info] sd30 is /pci@8,700000/usb@5,3/storage@3/disk@0,0
Apr 19 11:49:37 accelere12 genunix: [ID 408114 kern.info] /pci@8,700000/usb@5,3/storage@3/disk@0,0 (sd30) online


The above is an example of the end of the dmesg output.

3) Run
iostat -En
for the correct name of the disk on your system. Below is sample output:

c0t6d0          Soft Errors: 1 Hard Errors: 0 Transport Errors: 0
Vendor: TOSHIBA Product: DVD-ROM SD-M1401 Revision: 1009 Serial No:
Size: 0.00GB <0>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 1 Predictive Failure Analysis: 0
c3t0d0 Soft Errors: 1 Hard Errors: 0 Transport Errors: 0
Vendor: SAMSUNG Product: MP0402H Revision: DEF1 Serial No:
Size: 0.00GB <0>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 1 Predictive Failure Analysis: 0
c1t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: SEAGATE Product: ST336605FSUN36G Revision: 0438 Serial No: 0132P0FKEM
Size: 36.42GB <36418595328>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c1t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: NETAPP Product: X225_ST336704FC Revision: NA40 Serial No:
Size: 36.70GB <36700028928>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0

The format for the output for each device is:

deviceName Error info
Vendor and Product Info
Media Error information

ls -l /dev/dsk
to see whether the devicename matches the address in dmesg (
/pci@8,700000/usb@5,3...
)
In the above example, I checked
mount -v
to see which of the devices were already mounted. The only one not mounted already was c3t0d0

4)
mkdir /usbdrive


5)
mount -F pcfs /dev/dsk/c3t0d0s2:c /usbdrive

Check that the partition is mounted with mount -v

cN: the N'th disk controller installed in the system

tN: the SCSI TARGET id. Also used for atapi, firewire and usb mass
storage device, but it's always 0. Use to identify a specific disk
device on the disk controller.

dN: the SCSI LUN id. For atapi devices this is 0 for master device,
1 for the slave device. For usb mass storge devices this field could
contain a value > 0 - for example usb flash card readers often
present the slots for the various flash card formats as multiple LUNs. For simple usb flash memory sticks, this should be
LUN 0.

sN: the slice N from the SunOS disk label. On a media without a
disk label, slice 2 can be used to access all of the medias contents. Solaris SPARC supports slices s0 - s7, Solaris x86
slices s0 - s15.


pN: on Solaris x86 only: p0 is the whole disk (similar to slice s2,
when there's no SunOS disk label), p1 - p4 are the four primary
fdisk partitions.


The ":c" suffix assumes the media contains the fat filesystem in
a separate fdisk partition, searches the fdisk partition table for
the first partition containing a fat filesystem, and mounts the
filesystem from that partition. On Solaris x86, instead of using
"p0:c" you could also use one of the p1 - p4 device; but the pN
fdisk slices don't exist on Solaris SPARC.

Thursday, July 14, 2005

Installing Apache, PHP, and MySQL on Solaris

Apache

For Solaris:
  • Make sure gcc is installed along with the Sun developer packages and /usr/ccs/bin and /usr/local/bin are in the path.
  • To check whether the developer packages are installed, try 'pkginfo SUNWhea'. If it says package information not found, the packages are not installed.
If gcc is not installed:
  • Download the appropriate gcc package for your machine from sunfreeware.com.
  • Also download libiconv from sunfreeware.com
  • gunzip both
  • pkgadd -d filename for both gcc and libiconv - if you get an error about no space left on the device or unable to unpack datastream, scroll down further for a resolution.
  • pkginfo SUNWhea
    (This is one of the developer packages)
  • The Sun developer packages need to be installed and
    /usr/ccs/bin and /usr/local/bin must be in the path.
    # export PATH=$PATH:/usr/ccs/bin:/usr/local/bin

  • Links:
    http://www-uxsup.csx.cam.ac.uk/install/solaris/pkginst.html

    http://people.arsc.edu/~clark/sun/PkgAnalyze.html

  • If the developer packages are not installed, put the Solaris 9 cd in. Go to /cdrom and navigate the mess until you find the directory Product. Then type the following:

    # pkgadd -d . SUNWhea
  • Remember, you can check whether a package is installed by typing pkginfo pkgname

  • For gcc, install SUNWhea, SUNWsprot, SUNWlibm, SUNWbtool, SUNWarc (but I'm not sure if you need the last one).

If pkgadd fails with "unable to unpack datastream" and "No space left on device"
  • pkgadd -d gcc... fails with the following:

The following packages are available:
1 SMCgcc342 gcc
(sparc) 3.4.2

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: 1

Processing package instance from

gcc
(sparc) 3.4.2
cpio: Cannot write "reloc/include/c++/3.4.2/sparc-sun-solaris2.9/bits/stdc++.h.g
ch/O2g", errno 28, No space left on device

...

pkgadd: ERROR: attempt to process datastream failed
- process failed, exit code 24
pkgadd: ERROR: unable to unpack datastream

Installation of failed (internal error).
No changes were made to the system.

  • A df -k shows enough free space on the destination slice.
  • This means that /var is being filled up as pkgadd extracts into /var/tmp. The following page details this:
Links:
http://www.brandonhutchinson.com/No_space_left_on_device_when_installing_packages.html

  • The steps to take are as follows:
  • # pkgtrans packagename /path/to/large/file/system
  • # pkgadd -d /path/to/large/file/system
  • If it prompts you to install the package again after "Installation successful", just CTRL-C back to the prompt.
  • Then
    # rm -rf /path/to/large/file/system/$PGKNAME where pkgname=SMCgcc342 in this example

For SuSE:
I ran into a problem with some library missing but I forget which. I found an RPM however.

  • Unzip/untar tar zxvf httpd-* and cd http-*
  • ./configure --enable-so --enable-cgi --enable-ssl
Hopefully this works.

Links:
http://dan.drydog.com/apache2php.html


More Apache

  • make
  • make install
  • edit httpd.conf and set the servername
  • for solaris 9, set Group to nobody!

  • # Make sure there's only **1** line with this directive:
    LoadModule php4_module modules/libphp4.so

  • # Add index.php to your DirectoryIndex line:
    DirectoryIndex index.html index.php

# Uncomment one of the following (but not both)
# Use for PHP 4.x:
AddHandler php-script php
# Use for PHP 5.x:
#AddHandler php5-script php

AddType text/html php

# PHP Syntax Coloring
# (optional but useful for reading PHP source for debugging):
AddType application/x-httpd-php-source phps

PHP

  • gunzip php
  • ./configure --with-apxs2=/usr/local/apache2/bin/apxs --prefix=/usr/local/apache2/php --disable-cgi --with-mysql
if you get an error about libgcc_s.so.1
export LD_LIBRARY_PATH=/usr/local/lib

or whatever the path you have on your machine
  • # cp -p php.ini-recommended /usr/local/apache/php/php.ini

MySQL

Initialize the database:
  • copy over your database

    -or-
  • if you are not importing an existing database, the database needs to be initiliazed using the configure script or mysql_install_db in $mySQL/bin
  • groupadd mysql
  • useradd -c "MySQL Server" -d /dev/null -g mysql -s /bin/false mysql
  • chown -R mysql:mysql /usr/local/mysql
  • run using /mysqld --user=mysql &

Dumping an Existing MySQL database
mysqldump --user=blar --password=barf DBNAME > DBDUMP.sql

Loading the dump into MySQL
mysql --user=blar --password=barf < DBDUMP.sql

Dump the mysql database if you want user information to be transferred as well.

XRMS and phpMyAdmin

Set of both of these is fairly straight forward. Just untar xrms and phpmyadmin inside apache's document root, in this case in /usr/local/apache2/htdocs.

Modify vars.php for XRMS.

phpMyAdmin also has an ini file in its directory, which is commented well.

Labels

Blog Archive

Contributors