The target of this week’s assignment is to use a fuzzer to generate broken data, to feed those to a target program, and potentially find issues.
We will use two fuzzers:
Radamsa, created at the University of Oulu. It is a file-based fuzzer that takes in a set of valid example files (“valid cases”) and outputs an arbitrary number of invalid files. Radamsa doesn’t inject the bad data to the target software; this is something that would need to be done by its user. In this exercise, injection will be manual. In real life, you would need to automate it.
American Fuzzy Lop, or AFL, by Michael Zalewski, which is an instrumenting synthetic corpora fuzzer. It takes in a valid example and runs the target binary, evaluating the results, and then changes the input for maximum gains.
The assignment can target any software you want, but both fuzzers should be used as they demonstrate different fuzzing approaches.
If the target you select for the assignment is closed-source (binary only), you need to use AFL’s QEMU mode. That approach is not explained in these notes, but you can find more information from AFL docs.
WARNING: DO NOT OPEN THE FUZZ CASES WITH YOUR “PRODUCTION” SOFTWARE THAT YOU ARE RELYING ON. For example, if you choose to fuzz DICOM files as described in this assignment, do not open them with a system that is actually being used to diagnose or treat patients. Or, if you choose to fuzz more general picture formats, do not open them with your mobile phone or your image management application that contains your precious family pictures.
I’ve seen cases where this corrupts the metadata database or thumbnail cache, and in the worst case, you can lose everything. Always use a test setup and throwaway data. And never use someone else’s system or account for testing.
Note that if you have an antivirus software installed, it may react to your fuzz test cases by deleting or quarantining them, as they may flagged as false positives. Also, if you do this on a managed computer such as your work desktop, this may cause a remote virus alarm to sound at your IT department. This is obviously not optimal. However, I also do not recommend disabling your antivirus software, so the solution really is to use an environment that has no (need for an) antivirus software.
Preparation for fuzzing
Obtain Radamsa as a current Git snapshot from https://github.com/aoh/radamsa. You can build it by doing the following:
git clone https://github.com/aoh/radamsa.git
cd radamsa; make
which works at least on GNU/Linux and Mac OS X boxes with git installed. Yeah, it’s Lisp.
To obtain and compile AFL, follow the instructions at http://lcamtuf.coredump.cx/afl/QuickStartGuide.txt (get the tarball from http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz).
You need to compile your target with AFL. For the suggested targets, compilation with the exact command lines you need, are explained below. If you are fuzzing something else, or use the QEMU mode to fuzz a closed-source target, see AFL guides.
Selecting a target
For this assignment, you are free to try to fuzz whatever program, with whatever input your program takes. If you don’t have anything special that you know you want to torture, I propose fuzzing DICOM picture tools and libraries.
DICOM, Digital Imaging and COmmunications in Medicine, is a standard for medical imaging data storage and transfer. If you’ve ever undergone a modern X-ray, MRI, or CT scan, a DICOM file was probably generated and then shuffled around the hospital - to a imaging specialist, and to your doctor. It is still common to get the images for yourself on a CD-ROM. There’s a lot of software out there for processing DICOM - and some of it sits on computers that are close to the patient in a clinical setting. Even Adobe Photoshop reads DICOM.
There are many DICOM parsing libraries. In 2016, we tried to fuzz gdcm and DCMTK, and although they have clear quality and speed differences, they also seemed to be pretty robust. This year, I noticed that ImageMagick has DICOM support, so my examples use that. ImageMagick doesn’t probably try to parse all the DICOM fields, so I am not expecting fireworks, but perhaps the DICOM code paths are a bit less travelled… We’ll see.
The instructions below have been tested on a recent Xubuntu. There’s no reason why they wouldn’t work with almost any distro.
Acquiring DICOM sample files
There’s a website at https://www.dicomlibrary.com/ that has DICOM files “…for educational and scientific purposes”.
If you’ve been to an X-ray/CT/MRI, you can get samples from your healthcare provider too.
Fuzzing the target
ImageMagick is available as a precompiled package for Debian and Ubuntu. If you want to start with Radamsa, this option is likely to be your fast track to fuzzing.
sudo apt install imagemagick
The command that you can use is
convert, for example:
convert dicomfile.dcm pngfile.png
Fuzzing precompiled binaries with Radamsa
Take a moment to familiarise yourself with the Radamsa command line.
Take the valid sample files you have gathered and put them into one directory, e.g.,
valid-cases/. Create a new directory for fuzz test cases, e.g.,
fuzz-cases/. Then run:
bin/radamsa -n number-of-fuzz-cases-you-want -o 'fuzz-cases/fuzz-%n.%s' -r valid-cases/*.dcm
assuming your sample files have a
You could start with some hundreds of fuzz cases. In real life, you would use much larger numbers.
Then just write a small script to feed the test cases to the target binaries.
Compiling and fuzzing with AFL
Obtain and compile AFL as described earlier, and locate
afl-gcc, and replace its absolute path in the commands below.
Compiling ImageMagick with AFL instrumentation:
git clone https://github.com/ImageMagick/ImageMagick.git
CC=/absolute/path/to/afl-gcc ./configure --disable-shared
Put a good selection of valid cases into a directory and create an output directory. Then run AFL against the target command, e.g.:
afl-fuzz -i valid_cases_dir/ -o findings_dir/ /path/to/ImageMagick/utilities/magick convert @@ output.png
@@ is a placeholder for an input file name, which is supplied at runtime by AFL.
Reporting potentially exploitable crash bugs to developers (optional)
This section is optional.
If you need to report a crash bug to developers, the most important piece of information is the crashing test case. You can just take the crashing test case and use that, or if you’re using AFL, you can create a minimised test case that triggers the issue (look at the
afl-tmin tool). A minimised test case can be a fraction in size of the original file, and although it is likely to be completely against the input specification, it is probably easier for the developer to process. Also, if your test cases contained private information that you’re unwilling to provide to the developers - for example, in the case of DICOM files, patient images - this helps to clean that information out.
Other information that you may want to include are (of course) the exact version you’re running, your platform (
uname -a) and toolchain versions (e.g.,
g++ --version), the backtrace from
gdb, and in a corporate setting, if you feel you need some more “business priority” for the issue, output from the
exploitable plugin (although potentially inaccurate) may grease the management level wheels.
Using Address Sanitizer (optional)
This section is optional.
If you compile the target yourself, you might consider enabling Address Sanitizer (ASan) by providing
-fsanitize=address -fno-omit-frame-pointer options. This will help you to catch more issues with bad input handling. However, this is completely optional and voluntary.
Exploitability triage (optional)
This section is optional. You do not have to do this.
A crash doesn’t mean it’s a security bug. Once you have a crasher, there is a possibility that you’ve (for example) hit an assertion, or it’s a bug that is impossible to exploit.
You need to feed the crashing samples to the target and see what happens. One option is to run the target under
gdb, which allows you to see the crash much more clearly. To see the backtrace:
(gdb) run crashing-test-case-file-name (and wait for the crash)
You can usually see a bit about what’s going on when the crash happens.
If you have a large number of crashers and you want to quickly triage them, one option is using the
exploitable plugin for
gdb. You can install and use it as follows:
# apt install execstack python-setuptools (these are dependencies)
git clone https://github.com/jfoote/exploitable.git
python setup.py test (this tells you if the plugin has a chance of working)
(gdb) source exploitable/exploitable.py
(gdb) run crashing-test-case-file-name (and wait for the crash)
The plugin will show a rough exploitability assessment. The ‘Hash’ line is a unique fingerprint of the crash (based on call stack), and you can determine if the crash is unique by comparing the hashes.
For example, running the easily exploitable demo program from the lecture, I get
(gdb) exploitable Description: Access violation during return instruction Short description: ReturnAv (1/22) Hash: 0fa1c920f468682434123ec6508c5e75.02449cf2178ea872ebe3d090f0c6f41f Exploitability Classification: EXPLOITABLE Explanation: The target crashed on a return instruction, which likely indicates stack corruption. Other tags: PossibleStackCorruption (7/22), AccessViolation (21/22)
…which we already knew.
What to return
Note that you need to try out both Radamsa and AFL.
Provide a very short explanation (all together total less than 300 words) that explains what you fuzzed and how. This explanation should contain:
- What you selected as the target;
- The number of valid cases and how you selected (i.e., why did you decide to use those samples) and grouped them;
- For Radamsa, how many fuzz cases you created;
- For Radamsa, an explanation how you injected them to the target (depending on the target, this could be a shell script, shell commands, or a screenshot of a GUI application showing you inject the samples);
- For AFL, a screenshot of AFL executing against your chosen target;
- What was the most interesting result you observed (with each of the fuzzers - if you saw nothing, report that); and
- What would be your single most important next step to make fuzzing more effective, given this target. Think about a real-life software project, where security may not be the primary motivator, and longer-term maintenance.
For this assignment, just performing the fuzzing without discussing the potential enhancements (the last bullet, above) will earn a maximum score of 4/5. In order to obtain 5/5, you need to provide some perspective on how you would approach the task to make it more effective, and why you think it would be more effective. Please remember the maximum response length. There is no need for a preamble / intro / niceties, just the facts as listed above.