A Collection of UNIX Commands I Find Useful

1. Deleting recursive files & directory structure

find . -name "FILE-TO-FIND"-exec rm -rf {} \;

b. Deleting recursive files but leaving directory structure

find . -type f -name "FILE-TO-FIND" -exec rm -f {} \;

c. finding empty directories and deleting them

find . -type d -empty -exec rmdir {} \;

d. deleting files of a certain size

 find . -size <size>c -exec rm {} \; --where c=kb M=MB, etc

Note: if you want less than a specific size for <size> put (eg) -20M or more +20M

e. Removes all files older than and including <file>

find . ! -newer <file> -exec rm -rf {} \;

2. to display filename and line count minus 1 for csv files (well, any text file)

for i in `ls -1`; do echo -n "$i,"; j=`cat $i|wc -l`;j=`expr $j - 1`;echo $j; done


wc -l * | awk '{ if ( $2 != "total" ) printf"%s,%s\n",$2,$1-1}'

3. to remove whitespace and replace with underscores from file names recursively

NOTE: this method is currently imperfect if subdirectories also contain white space

for i in `find . -name '* *' | tr ' ' '~'`; do mv "$(echo $i| tr '~' ' '|sed 's/ /\\ /g')" "$(echo $i | tr '~' '_')"; done

4. how to flatten a directory structure

find . -type f -name "*mp3" -print0 | xargs -0 -J% mv % <destination_directory>

This example scans a directory tree for all mp3 files and moves them to a single directory

5. display full paths to files for recursive directories

find `pwd` -name "*.*"

6. copying a disk to a disk image using dd

dd bs=512 if=/dev/rXX# of=/some_dir/foo.dmg conv=noerror,sync

NOTE: conv=noerror,sync skips bad blocks and fills the target with nulls, thereby making it an excellent way to recover a broken disk, so long as it mounts.

The basic syntax for dd is:


b. making an ISO image using dd

To make an ISO from your CD/DVD, place the media in your drive but do not mount it. If it automounts, unmount it.

 dd if=/dev/dvd of=dvd.iso    # for dvd

dd if=/dev/cdrom of=cd.iso bs=2048  # for cdrom

dd if=/dev/scd0 of=cd.iso    # if cdrom is scsi

c. burning disk image to disk / usb using dd

This is useful for making bootable USB Sticks of OS iso or dmg images

dd if=<path_to_image>/<imagefile.dmg of=/dev/rdisk8 bs=1m

NOTE: Remember to write it to the raw disk.

7. Symlinks 

a. Find broken symlinks

find . -type l -print -exec file {} +|grep broken

b. show target and source of symlinks in a dir

find . -type l -exec readlink {} \; -exec echo {} \;|awk '{printf"%s - ", $0 ; getline ; print $0 }'

8. Remove spaces from a file name recursively and replace with underscores

find . -name '* *' | while read file;


target=`echo "$file" | sed 's/ /_/g'`;

echo "Renaming '$file' to '$target'";

mv "$file" "$target";


flattening a dir - note -print0 is mac specific to handle spaces in file names


find . -type f -name "*vsd" -print0|xargs -0 -J% cp % /Volumes/Charles05/charles/cw/customers/vertex/One/design/visio_vdx 

gzip and find


find . -type f -name "archlog*.dbf" -exec gzip {} \; &        # find everything

called archlog*.dbf and then compress it


find . -type f -name "archlog*.dbf" -mtime +2  -exec gzip {} \; &    # find

everything older than two days and compress it


gzip archlog*.dbf &      # Compress all logs with this name .


cd <dir>

for f in `ls arch*.dbf`


if [ -z `fuser $f` ] ; then

   gzip $f


   echo "$f is in use"


or this:

cd <archdir>

find . -type f ! -newer `ls -1tr | tail -1` -xargs gzip {} \;

gzip a directory to a single file


tar cvf - <directory to be zipped>|gzip > <filename of single file to compress to>.gz

grepping for multiple words use egrep


egrep "(<word1>|<word2>)" filename

e.g. egrep "(CREATE|DROP)" IS001A.sql

full fsck


fsck -F vxfs -o full -y <File system>

e.g. fsck -F vxfs -o full -y /dev/vx/dsk/bcv1OtherDbs-livedg/OtherDbs-orabin-9i

Keeping the sudo token active


while true


sudo ssh crockofshi7

sleep 180


while true; do sudo ssh thingy; sleep 240; done

Finding how many processors



Finding out where the large files are


du -sk *



killing processes by name

pkill -9 <string>

adding users


useradd -g 10 -d /export/home/charlesisapoof -m -s /bin/ksh -c "Charles Webster" websterc



How to push a whole file system from one machine to another through ssh

tar cf - * | ssh -c blowfish file_share@tiger "cd /Users/file_share/jad; tar xf -"

full path listings


 for i in `ls -1`;do echo `pwd`\/$i; done

or in awk (courtesy of Dave Leather)

ls|awk '{print "'`pwd`'/" $1}

if you want to exclude certain files based on file name use this (example here is "small")

ls *.jpg|awk '{if ( $0 !~ /small/) print "'`pwd`'/" $1}

(to do this on inclusion rather than exclusion drop the !)


Sun inventory



Making an encrypted disk image (sparsefile)


hdiutil create -size 62gb -encryption -type SPARSEBUNDLE -fs "Case-sensitive Journaled HFS+" -volname "Charles08"  /Volumes/CSW_USB_08/Charles08.sparsebundle -spotlight

converting sparseimage to sparsebundle


hdiutil convert -format UDSB -o <target.sparsebundle> <source.sparseimage>


Solaris - Displaying kernel parameters not listed is /etc/system



script to read a log file showing the current and a backup of a file and report the differences


ls -t /u01/app/oracle/product/istore/cwciteappl/admin/CWCITE_ordsvywat-sun-09/log/*/adconfig.log | head -1 | \

awk '{ printf" grep backup %s\n",$0 }' |sh  |\

awk '{print "cmp " $3 " " $5 }' | sh

LINUX with HP RAID Controller - Checking status of HP Raid Controller


/usr/sbin/hpacucli controller slot=0 pd all show

mounting a SMB / NFS drive


mount -t smbfs <smb_drive_to_mount_with_auth> <local_mont_location>

mount -t smbfs //cwebster@bdc.localnet/CSW01 /<path_to_mount_point/<mount_point>

on osx


mount_smbfs //cwebster@bdc.localnet/CSW01 /<path_to_mount_point/<mount_point>

#to mount nfs stipulate nfs rather than smbfs

Encrypt a file using OpenSSL 


This is a pretty simple way to encrypt a single file so that it is nearly impossible for others to read, and it will require a password to access again. You’ll need to use the OpenSSL technology via the command line for this to work.

Encrypt the file:

$ openssl enc -e -a -salt -bf -in <plain_filename.txt> -out <encrypt_filename.txt>

enter bf-cbc encryption password:

Verifying password - enter bf-cbc encryption password:

decrypt file:

$ openssl enc -d -a -bf -in <encrypt_filename.txt> -out <plain_filename.txt>

enter bf-cbc decryption password:

--find all the instances of a pattern  in excel files and pipe them to a file

find . -name '*.xls'

--find all the instances of a pattern  in excel files and pipe them to a file with associated file name

find . -name '*.xls' -exec grep -b -n -i vtx {} /dev/null ';' | strings | egrep 'vx|VX|.xls:' | awk '{if ( $0 ~ /.xls:/ ) { file=sprintf("%s",substr($0,1,index($0,":"))) ;getline} ; print file,$0 }'

Port Scanning


scan a range of servers for a specific port


for i in $(seq 160 170); do nc -v -n -z -w 1 10.196.52.$i 389; done

Scan a server for a specific port


nc -z gbcwswiep005.ad.plc.cwintra.com 100-700

Finding other machines on your network (same broadcast domain)


--find the broadcast address -- ifconfig -a | grep broadcast

--ping it --ping -i 5 -c 2 <broadcast_address>

--alternatively arp -a will give you all the machines your computer has perviously interacted with.

© Charles Webster 2016