Vulnerability disclosures rarely include enough technical detail to reproduce the exploit. This is a good thing. It wouldn’t do to arm every script kiddie with exact details of how to write an exploit with every disclosure. However, there are times when someone like an application security engineer or security researcher need to “reverse engineer” the disclosure to reconstruct the technical detail in order to fully understand the vulnerability or write an exploit to test systems for weakness.
Why is this useful?
When a vulnerability is found in an open source project, it’s often in a library. A vulnerability in a library could potentially affect many applications, but it’s notoriously difficult to tell if an application is actually vulnerable just by using the vulnerable library. The application may not actually use any of the vulnerable code or be configured in such a way as to be vulnerable. Of course, this itself might not be obvious because the vulnerability disclosure may not fully explain these details, or it may even be wrong and say that a library is vulnerable when the vulnerable code is in a dependency of the library! A company may be hesitant to upgrade a vulnerable library because of the cost, especially at large companies which have a lot of procedures in place around code changes. We’ve had customers tell us:
“You tell us we have this vulnerability, but we looked into it and aren’t using any of the vulnerable code, plus it would cost a million dollars to upgrade.” - Random J. Customer
This really surprised me when I first heard it, but it made sense after I remembered the time I tried to upgrade a gem in a large Rails project with lots of dependencies. Let me tell you, that is a path which leads quickly to Dependency hell.
The cost of upgrading is only worth it if the system is actually vulnerable, and the fastest, most direct way to test if a system is vulnerable is to hit it with the actual exploit. This type of test exploit is usually called a PoC (proof of concept). To make a PoC, you really have to understand the nitty-gritty details of the vulnerability. And if a PoC isn’t possible, understanding the vulnerability will help you determine if an application is truly vulnerable.
Of course, there are also secondary advantages such as education for engineers and security researchers. Understanding the vulnerability can help a developer avoid making the same mistakes. Researchers can use the knowledge to search for and discover (and responsibly disclose) similar vulnerabilities.
How it’s Done
There are three main challenges to reversing a vulnerability:
- Find the fix
- Understand the fix
- Create a PoC
Finding the Fix
This is the most important and complex of all three steps. Sometimes it’s easy, and the public disclosure links right to the commit hash, and sometimes it’s hard because you spend hours digging through a repository’s commit messages only to find a commit with the message “update pom.xml” that does indeed update the pom file but also sneaks in the actual fix (true story). Some open source groups such as Apache are quite good at linking their vulnerability disclosures to fix commit or at least a link to an issue tracker like JIRA with more details and usually links back to commits. On the other hand, some open source communities go out of their way to hide the fix because they believe it would help the bad guys.
While it’s true that the fix would allow attackers to more easily understand the details of the exploit, I personally believe that any attacker sophisticated and motivated enough to analyze the source code to extrapolate a weaponized exploit is going to find the fix with or without help. Making the fix obvious makes a lot of legitimate endeavors easier, such as allowing package maintainers to back-port fixes to older, unmaintained versions and, of course, all the reasons I mentioned at the beginning of the post.
The first place to look for the fix is the disclosure. Read it carefully and understand it. Try and read the documentation about any features it mentions. This context will help you find and understand the fix commit, which is necessary to make the PoC. The disclosure will likely have some good keywords for searching if the exact fix commit or relevant issue tracker IDs aren’t referenced.
Let’s assume there is a disclosure which has very little information and you must start searching for the fix yourself. One critical piece of information should already be obvious: the fixed version. You at least know one of the commits in version has the fix. Unfortunately, there may be several hundred commits and each one is changing all kinds of stuff and you have no idea what’s going on in the code base. You need to filter down the commits to look through. First, read the disclosure, and keep this stuff in mind:
- Is the fix in code or configuration?
- Are any classes specifically named?
- Are any features specifically named?
- Is there a mantainer’s name associated with the advisory? Maybe they did the fix also.
Sometimes there is only one or two committers who fix security bugs. - The fix happened before the public disclosure. Any commit after the advisory won’t contain the fix.
- If the fixed version is a hot fix release (e.g. 1.2.3 -> 1.2.3.1), there are probably only a few changes to look through.
Next, try and get lucky by searching Google. Maybe someone else has already done all the hard work and there’s a blog post and a working PoC or Metasploit module just waiting for you. Fat chance, though. Explaining a vulnerability is hard and not many just give them out for free.
If the open source project has an issue tracker, try searching that. Use keywords from the advisory or search for issues created within a few weeks before the advisory was published. Depending on how strict the maintainers are about using their issue tracker, it may also be possible to filter for affected versions.
You can also search the git logs. My favorite ways to search are:
1 | git log --all -i --grep 'exploit keyword' |
The --all searches through all branches. This can be necessary when the fix only appears on a certain branch, since you probably wont know which one. And the -i tells grep to be case insensitive.
A great technique is to compare the commits between the fixed version and the version right before that. This can be accessed on GitHub by going to the Releases for a project. For example, Apache Storm Releases.
First, select the tag before the fixed version. Let’s say v0.10.0-beta is the tag right before the fix and v0.10.0-beta1 contains the fix. There is a link on the release page which says “1892 commits to master since this tag”.
That link will compare that release with master. In this example, it’s https://github.com/apache/storm/compare/v0.10.0-beta1...master. Notice the v0.10.0-beta...master. Change this to v0.10.0-beta...v0.10.0-beta1 to get this link: https://github.com/apache/storm/compare/v0.10.0-beta...v0.10.0-beta1.
Understanding the Fix
Ok, so you found the fix commit, or what you think and hope is the fix commit. Great. Now you just need to understand what the change was and how it affected the code. For this, you are going to want to setup an environment for the project. You want to be able to compile, run and debug whatever it is.
Apart from this, there’s not much I can say. Read the commit message carefully. Good luck!
Creating a PoC
If you didn’t need a working environment to understand the fix, you will likely need one to create the PoC. It’s almost impossible to simply read the code and produce a working PoC. It will take several iterations.
Many PoCs are written in Ruby or Python because they’re flexible and don’t require a lot of structure and planning to get something complex up and running. You could even use the metasploit framework.
Whatever you do, just remember the PoC won’t make sense to you a few days or weeks from now. Do yourself a favor and document any of the weird strangeness you encounter for future reference. Since your PoC code is exploiting an edge case that eluded developers, there is likely going to be some strangeness. Writing clear and useful comments will also help you understand what is going on.
Also, if you want your PoC to look legit or just for bonus points, use the standard “[*]” and “[!]” notation for output. For example:
1 | [*] Connecting to server ... |