Some Useful Linux or Unix Commands

• du -h /home/richard - this returns the size of the directory in a format readable by humans
• http://www.apmaths.uwo.ca/~xli/vim/quickstart.html for the **** vim
• http://www.unixgeeks.org/security/newbie/unix/cron-1.html for a good cron job tutorial (this wiki is now saved regularly ; )
• grep and pipe
  • man grep | grep reg
    
    this returns each line in the manual entry of grep that contains the string \"reg\" somewhere in this line
  • cat dict.lsp | grep PRE | cat > preps.lsp
    
    prints out the contents of the file dict.lsp, the pipe takes these arguments and gives them to grep as input, grep then filters out all the lines that contain the string PRE and gives this input to cat which opens a file and writes all these lines into the file preps.lsp
• cut
  • Example of 'cut'
    cat preps.lsp | cut -d \" -f2 | cat > onlypreps.txt
    
    Explanation: with the first part I output the file preps.lsp to standard output. this is the input to the cut command. This cuts each line into parts seperated by the delimiter of the quoting symbol (needs to be behind a backslash). One can think of the result as a little hash from which I take the second field with the -f2 command. The last pipe returns the output to the cat command which prints the output to the file onlypreps.txt
  • Using this and the second example of grep, I now have a list with all the prepositions of my dictionary, which has 120000 lines looking like this:
    (setf(gethash "avec" *dict*)'((ADV "avec" NIL  NIL  ((NIL )) 159.95)(PRE "avec" NIL  NIL  ((NIL )) 7378.48)))
    (setf(gethash "ave" *dict*)'((NOM "ave" m NIL  ((NIL )) 5.55)))
    (setf(gethash "avelines" *dict*)'((NOM "aveline" f p ((NIL )) 0.07)))
    (setf(gethash "avenant" *dict*)'((ADJ "avenant" m s ((NIL )) 1.3)(NOM "avenant" m s ((NIL )) 0.94)))
  • The first grep step would return only the line:
    (setf(gethash "avec" *dict*)'((ADV "avec" NIL  NIL  ((NIL )) 159.95)(PRE "avec" NIL  NIL  ((NIL )) 7378.48)))
    and the second then:
  • The cut command returns: 'avec'
• tar
  • This code packs all files in a directory into a g-zipped tar package, splits this tar package into multiple files of size 500MB and names them accordingly
    tar cz hugeDirectory/ | split -b500m - TaredAndZippedDirectory.tgz.split.
    
• scp
  • used for securely copying folders from one computer to another
  • scp -r folderName rsocher@sub.domain.edu:~/
    
  • this copies the entire folder
• xargs - apply a command on every single file in folder and its subfolders
  • find folder/ -maxdepth 2 -type f | grep -E "this|or|that|in|filename" | xargs -l ./getFeat other args
    
• -exec if you want to fork the piped program for each input, this is especially helpful, if some of the inputs make the program fail. Then xargs would stop completely, but -exec would continue:
  • time find positives/new* -maxdepth 2 -type f -exec ./getFeat 2 1 {} \; >> goKart.txt
    
• screen http://www.linux.com/articles/56443
  • screen # to open a new session
    Ctrl-a d # to detach, now you can log out of the machine and then log back into this session by:
    screen -r # to reattach
    screen # open another session
    screen -list # list al open sessions
    screen -r xxx # reattach screen with the number xxx
    screen -S name # opens a scrren with the name for easier reference later
    
• nohup http://en.wikipedia.org/wiki/Nohup - simple alternative to screen for some cases
• Perl - How to install new CPAN modules?
  • perl -MCPAN -e shell # go to CPAN install mode
    install Bundle::CPAN # update CPAN
    reload cpan
    install Set::Scalar
    
• tail - print the end of the file and updates it, whenever the file changes, nice to watch log files
  • tail -f filename
    

Bash Scripts

• This bash script deletes all files in a directory, except the first 100 files
  • #!/bin/bash
    total=`ls $1 | wc -l`
    deleteTail=`expr $total - 100`
    rm -v `find $1 | tail -n $deleteTail`
    
  • call it after copying it to a file keepOnlyFirst100.sh by: ./keepOnly100.sh directoryName
  • if you want to call it on all directories in your current directory, do:
  • find . -maxdepth 1 -type d -exec ./keepOnlyFirst100.sh '{}' \;
    
• or if you have a script that you want to apply to all txt files and which has 3 inupts, a number and the input file and output file:
  • find *.txt -maxdepth 1 -exec ./avgimg '400' '{}' '{}.jpg' \;