Wednesday, 31 July 2013

Lines of the dead

Your humble zombie has been casting around for another topic to blog about.  I found a number of things but they didn't warrant a single blog post.  Therefore this post will cover several single line commands that are quite simple but powerful at the same time.   Hopefully they will illustrate the power of the GNU/Linux CLI, and maybe prove useful to you in your work.

Recovering a single record from the MFT is easily do-able, but why would you want to, after all the istat command in the sleuthkit can parse any MFT records (and in the most useful and understandable way amongst all the forensic tools that I have looked at)?   Well, MFT parsing tools generally report on the data in the "allocated" area of the MFT record, they don't report the residual data in the record.   The residual data could contain information useful to your investigation.   It is important to understand that when the data is written to an MFT record for a NEW file, it zeros out the data after the end of record marker.   Therefore, any residual data that you find in an MFT record MUST have once been part of the allocated area of the record.    So, how did I extract the records?   A simple one-line command for each record (I am using an arbitrary record number in this example) like this:
icat -o 2048 bad-guy.E01 0 | dd bs=1024 count=1 skip=9846 > MFT9846.dat

The first part before the pipe simply output the contents of record 0 on the partition starting at sector 2048 on my disk image.   Record 0 on an NTFS file system is the MFT, so the entire MFT is output.  We then pipe the output to the dd command, specifying a block size of 1024 bytes (the size of each record, and skipping a number of records until we get to the one want.  The output is redirected to a file - job done!

Here is a way to identify files that contain an arbitrary number of a particular string.   Another example of how this might be useful:
In several examinations I have found web-pages in the web cache that contain chat records, these are generally from websites that use flash based chat applications.   I noted that the chat messages were generally preceded by the string "said:".   Thus you would see the username followed by the string "said: " followed by the message.   I therefore automate the process of searching for web pages that contain SEVERAL instances of the string "said: ".    Obviously if we looked for web pages that contained a single instance of the string "said: " we are going to get a number of false positives, as that string might appear in a web based story, or very frequently a news story quoting a source.
So, we could find all the web-pages in the mounted file system, remembering that if it is a Windows box being investigated there is a likelihood that there will be white space in file names and file paths so we need to set our Internal Field Separator environmental variable to new lines only, like this:
Thats IFS=' followed by a press of the return key followed by a single ' character.   Theoretically you should be able to type: IFS='\n' but it does seem to work for me, thus I explicitly use the return key.

find /disk/image -type f -exec file {} \; | awk -F: '$2 ~ /HTML document/ {print $1}' > /tmp/livewebs.txt

This command conducts a file signature check on the file in the live file system and looks at the description field in the output for the string "HTML document" if there is a match the file path and file name are sent to a file in my /tmp directory.

I can then process my list of files like this:

cat /tmp/livewebs.txt | while read LWC ; do egrep -H -c -e " said:" -e " says:" -e "msgcontent" $LWC | awk -F :  '$2 > 2 { print $1 }' > /tmp/livewebchat.txt ; done

The above command reads my list of files, line by line, it searches for 3 regular expressions ( said:, says: and msgcontent).   The -H option for egrep reports the file path and file name, the -c option reports the number of matches for each file.   So, a list of file paths/names are generated followed by a colon, followed by the number of matched regular expressions for each file.   The result is piped to awk, the field separator is set to a colon (giving us 2 fields).  The second field is checked to see if the number of matches is greater than 2, if they are the first field is (the file name/path) is sent to a new file in the /tmp directory.   I could then use that new file to copy out all of the files:

cat /tmp/livewebchat.txt | while read i ; do cp $i /home/forensicotd/cases/badguy/webchat ; done 

Obviously I would need to do that again with any deleted files I've recovered with the tsk_recover command or any unallocated web pages I have recovered with photorec.   I would need to do something similar with gzipped files as well, as they could be compressed web pages - so use zgrep instead of egrep.   Remember that simply coping files out is a bad idea - you need to use the FUNCTION I posted to ensure that files don't get over-written in the event of a file name collision.

Another pain in dealing with web-pages is actually viewing them.  You REALLY shouldn't be hooked up to the 'net whilst you are doing forensics, if you are, then you are doing it WRONG!  But browsers often struggle to load raw web-pages when they are trying to load content coded into the html in your web page.    So, one thing we could do with the web pages listed in our livewebchat.txt file is remove all the html tag, effectively converting them to text files that we can view easily in a text editor, or the terminal.   We can do that with the html2text program.  Even better, we can output the full path and filename at the top of our converted webpage so that if we find anything interesting we know the exact file path that the file came from.  Here is a function that I wrote to convert the web-page to text, output the file path/name and check if the file name exists and adjusting the file name to prevent it over-writing a file with an identical file name:

suswebs () {
filename=`basename "$1"`
msg=`html2text $1`
while [ -e ${WDIR}/"${filename}" ]; do
        filename=`echo "$filename" | perl -pe 's/(\[(\d+)\])?(\..*)?$/"[".(1+$2)."]$3"/e;'`
echo "${filepath}${msg}" > ${WDIR}/"${filename}.txt"

To use the function you need to set the WDIR variable to the location where you want your processed web-pages to go to, so something like this:
cat /tmp/livewebchat.txt | while read l ; do suswebs $l ; done

Obviously you can add all of these commands to your linux preview disk.

Monday, 1 July 2013

Maintaining the dead

In this post I will walk you through working with the new overlayfs feature in recent Linux kernels, and how to use that to make changes to a Linux Live distro truly permanent.

For most of my forensic previewing I use the CAINE forensic GNU/Linux distro.  You can fully customise the distribution to fit your needs.  Most of the stuff I blog about I have added to my distro to automate the previewing of suspect devices.

The CAINE guys recently released a new version of, called PULSAR.  They have necessarily had to make some changes due to how Ubuntu is now packaged (CAINE is based on the Ubuntu distro).  Previously, you could download an .iso to burn to CD and a DD image to copy to a thumbdrive.   One of the major changes is that the version designed to be installed to a USB device (or indeed a hard disk on your workstation) comes as an .iso.   I did some major surgery to the previous versions then used the remastersys scripts bundled in CAINE to generate my own customised .iso.   Obviously, it isn't that simple any more, so I thought I would walk you through making persistent changes to the latest CAINE distro.

The basic steps to customising CAINE are:
Download NBCAINE 4.0
Install to thumbdrive
Make your changes to the distro
Merge your changes into the system
Create your new .iso

The first thing you need to do is download NBCAINE 4.0 from the CAINE website.  You then need to use live-disk installer to install the .iso to your thumbdrive.  I use UNETBOOTIN for this, but there are other installers out there for you to experiment with.  You will also need a clean USB stick, formatted as FAT32.  So all you need to do now is plug in your thumbdrive, launch unetbootin and configure it like this:

You need to make sure you have selected Ubuntu 12.04 Live as your Distribution.  Then select the path to your nbcaine .iso image.   You then need to configure how much space your require to preserve files across reboots.  I make lots of changes so I select 2000 MB - you will need an 8 GB thumbdrive if you go this big.  I then select the partition on my thumb drive that the distro is going to be installed to.
Once you click on "OK" the installation will start. Once complete, you can reboot, select your thumb drive as your boot device to boot into the CAINE environment.

One of the big differences you will find in CAINE 4.0 is that the entire structure is radically different from version 2.0.  Previously, when using the dd image of caine, you would have found the expected linux directory tree in the root of the partition, directories such as bin, boot,  home, var, usr, etc.  That directory structure does still exist, however it is contained inside a squashfs file system, that is decompressed at boot.   The important files are contained within the "casper" directory in the root of your .iso.  We should have a look at the root directory to see what's there - this will help us understand how the CAINE system works:

-rw-r--r--  1 fotd fotd  176 2013-03-15 17:47 autorun.inf                                                                                                      drwx------  2 fotd fotd 4.0K 2013-06-28 16:07 casper
-rw-r--r--  1 fotd fotd  1.9G 2013-06-28 16:49 casper-rw
drwx------  2 fotd fotd 4.0K 2013-06-26 13:31 .disk
drwx------  4 fotd fotd 4.0K 2013-06-26 13:31 FtkImager
drwx------  2 fotd fotd 4.0K 2013-06-26 13:34 install
drwx------  2 fotd fotd 4.0K 2013-06-26 13:34 isolinux
drwx------  2 fotd fotd 4.0K 2013-06-26 13:34 lang
-r--r--r--  1 fotd fotd  32K 2013-06-26 13:35 ldlinux.sys
-rw-r--r--  1 fotd fotd  52K 2013-03-16 16:38 md5sum.txt
-rw-r--r--  1 fotd fotd  55K 2013-06-26 13:35 menu.c32
-rw-r--r--  1 fotd fotd 1.4K 2013-03-15 17:56 NirLauncher.cfg
-rwxr-xr-x  1 fotd fotd 102K 2013-03-15 17:47 NirLauncher.exe
drwx------  2 fotd fotd  36K 2013-06-26 13:31 NirSoft
drwx------  2 fotd fotd 4.0K 2013-06-26 13:35 piriform
drwx------  2 fotd fotd 4.0K 2013-06-26 13:35 preseed
-rw-r--r--  1 fotd fotd  192 2013-03-16 15:51 README.diskdefines
drwx------  2 fotd fotd 8.0K 2013-06-26 13:35 sysinternals
-rw-r--r--  1 fotd fotd 1.7K 2013-06-26 13:35 syslinux.cfg
-rw-r--r--  1 fotd fotd  25K 2013-06-26 13:35 ubnfilel.txt
-rw-r--r--  1 fotd fotd  20M 2013-03-16 15:53 ubninit
-rw-r--r--  1 fotd fotd 4.7M 2013-03-16 15:53 ubnkern
-rw-r--r--  1 fotd fotd 1.6K 2013-06-26 13:31 ubnpathl.txt
-rw-r--r--  1 fotd fotd    0 2013-03-16 16:37 ubuntu
drwx------  7 fotd fotd 4.0K 2013-06-26 13:35 utilities

Many of the above files are for the live incident response side of things.  The structures that we are interested in are the casper-rw file and the casper directory.  Remember when we used unetbooting and were asked how much space we wanted to reserved for persistent changes?  Whatever value you enter, the casper-rw file is created with a size corresponding to that value.  In our case we specified 2000 MB, which is about 1.9 GB, hence our casper-rw file is 1.9 GB.   It is this file that captures and stores any changes we make to the CAINE system.   Lets have a look inside the "casper" directory:

-rw-r--r--  1 fotd fotd  62K 2013-06-28 16:39 filesystem.manifest
-rw-r--r--  1 fotd fotd  62K 2013-06-28 16:39 filesystem.manifest-desktop
-rw-r--r--  1 fotd fotd   10 2013-06-28 16:39 filesystem.size
-rw-r--r--  1 fotd fotd 1.5G 2013-06-28 16:41 filesystem.squashfs
-rw-r--r--  1 fotd fotd  20M 2013-03-16 15:53 initrd.gz
-rw-r--r--  1 fotd fotd  192 2013-03-16 15:51 README.diskdefines
-rw-r--r--  1 fotd fotd 4.7M 2013-03-16 15:53 vmlinuz

The main file of interest here is the filesystem.squashfs.   This file contains the compressed Linux file system, it gets decompressed and mounted read-only at boot.   The previously described casper-rw is an ext2 file system that is mounted read-write at boot - it retains all the changes that are made.  Using the new overlayfs system, the casper-rw overlays the decompressed filesystem.squashfs, but it is interpreted by the O/S as a single unified file system.  We should bear in mind that the files "filesystem.size", "filesystem.manifest" (which lists all the installed packages) and "filesystem.manifest-desktop" have to accurately reflect various properties of the filesystem.squashfs file.  Thus, if we add programs to our installation we have got to update all of the "filesystem.*" files.

Using overlayfs is useful for retaining changes over reboots, however it does mean that the system is somewhat bloated.   What we really need to do is merge the changes in the casper-rw file system down into the compressed filesystem.squashfs, then we don't need our large 2000MB casper-rw file any more (or at least can use a much smaller file once we have made our major changes).

So, I have figure out how to do this...and it isn't actually that difficult.  All we need is to make sure we have the squashfs tools installed and a recent Linux kernel that supports overlayfs.   To check if your kernel supports overlayfs, simply have a look a look at your  /proc/filesystems file and see if it is listed.  If not you will need to install the overlayroot package, which is hopefully in your package manager.      So let's go through this step-by-step. We've created our bootable thumb drive, booted into that environment, and made all the changes that we want to make.  So shutdown and reboot into your Linux environment, plug in your CAINE thumbdrive and mount the partition in read write mode.

Change into a working directory in your linux distro.  We are going to create two mount points, mount our filesystem.squashfs and casper-rw file systems, then overlay one over the other with the overlayfs before creating a new filesystem.squashfs.   So, lets assume my thumb drive is mounted at /media/usb, now we create our mount points in our working directory:
mkdir caine-ro
mkdir caine-rw

Now we mount our casper-rw file in our caine-rw mount point:
sudo mount -o loop /media/usb/casper-rw caine-rw
We mount our filesystem.squashfs in our caine-ro mount point:
sudo mount -o loop /media/usb/casper/filesystem.squashfs caine-ro

Now we need to overlay one file system over the other using the overlayfs system.  Conceptually, overlayfs has an "upperdir" which takes precedence and a "lowerdir".  Obviously the "upperdir" gets overlaid on top of the "lowerdir", in our scenario the caine-rw is our "upperdir" and caine-ro is our "lowerdir".

Now we can do our file system overlaying:
mount -t overlayfs caine-ro -o lowerdir=caine-ro,upperdir=caine-rw caine-rw
We now have our two file systems overlayed with each other, we can now make a new filesystem.squashfs to replace the one on our thumb drive...but at which mount point is our unified filesystem from which we need to create our new filesystem.squashfs?  Remember it is the "upperdir" that takes precedence, which in our case is caine-rw.  So we need to create our new filesystem.squashfs from that mountpoint, we can do that like this:
sudo mksquashfs caine-rw/ filesystem.squashfs

We now have our replacement filesystem.squashfs that contains all our changes.
We now need to update the rest of the "filesystem.*" files to reflect changes in our filesystem.squashfs.  Lets update our filesystem.manifest:
sudo chroot caine-rw/ dpkg-query -W --showformat='${Package} ${Version}\n' > filesystem.manifest

Now we can update our filesystem.manifest-desktop with this commands
cp filesystem.manifest filesystem.manifest-desktop

Unmount our mounted filesystems:
sudo umount caine-r*

Finally, we can update the filesystem.size file - we'll need to mount our NEW filesystem.squashfs to do this:
sudo mount -o loop filesystem.squashfs caine-ro
printf $(sudo du -sx --block-size=1 caine-ro | cut -f1) > filesystem.size

At this point we have everything we need in our working directory, we just need to copy all of the "filesystem.*" files to our mounted thumb drive containing our CAINE system.  Remember they need to go into the "casper" directory:

sudo cp filesystem.* /media/usb/casper/

Let's unmount our filesystem.squashfs:
sudo umount caine-ro

To finish up we need to go and work on our mounted thumbdrive.

We can make a new (much smaller) casper-rw to hold any minor future changes we want to make.   We need to delete the existing, casper-rw, create a new one then format it - we'll use ext2 in our example:

sudo rm /media/usb/casper-rw
sudo dd if=/dev/zero of=/media/usb/casper-rw bs=1M count=300
sudo mkfs.ext2 -F /media/usb/casper-rw

Finally we need to make a new md5sum.txt file to reflect our changes:
cd /media/usb
sudo find . -type f -exec md5sum {} \; > md5sum.txt

All done!  Remember to unmount all your various mounted filesystems relating to your work with the CAINE installation.