Twelve Days of ZAPmas - Day 11 - ZAP impressions from a Burp user

It probably seems a bit odd to do this on Day 11 and not at the end of the series, but I have one more useful bit that I really wanted to save for last. For the uninitiated, ZAP and Burp Suite are two tools that mostly fill the same role. ZAP is a FOSS project supported by OWASP, while Burp Suite is a commercial tool. I will be comparing against Burp Suite Professional, since that’s what I’ve used most of the time, but I also want to note that there’s a free Community Edition. I came into this series having used Burp extensively over many years, but barely touched ZAP. So I wanted to share my impressions on how they compare, bearing in mind that Burp is still much more familiar to me. One limitation in the scope of this comparison is that while both apps support extensions, I’m going to focus on the stock feature sets.

Maintaining Scope and why ZAP Contexts are Awesome

While Burp has a concept of in-scope and out-of-scope, ZAP contexts add an additional layer of flexibility that Burp doesn’t really have an analog for. Combined with how easy ZAP makes it to add include and exclude rules without writing regexes (which I enjoy, but certainly aren’t for everyone), I think ZAP really does a fantastic job in this area. I never had an issue with how Burp’s scope worked, but I really think ZAP’s approach is clearly superior here. The Protected Mode in ZAP is the icing on the cake, since it prevents even rudimentary tampering against out-of-scope resources.

Manual Testing with Proxy, Fuzzer and Request Editor

Neither Burp nor ZAP struggle to provide this set of tools. Both proxies work similarly, capturing and filling up history and populating a site map. Both support some form of commenting and highlighting on requests in the proxy history. There are differences in layout and slight differences in the information they present, but overall I would consider them approximately at parity.

The replaying and tampering tools (ZAP Request Editor + Requester and Burp Repeater) take slightly different approaches. Burp’s repeated requests don’t appear in the proxy history, but are kept on multiple, renamable tabs in the tool itself. ZAP’s Request Editor includes the requests in an easily identifiable way in the proxy history instead. Additionally, the Requester tool works almost identically to Repeater. I like the extra flexibility in ZAP’s approach, while Burp has a comfortable familiarity for me.

ZAP Fuzzer and Burp Intruder take slightly different approaches to the user-interface side of setting up fuzzing. Burp presents a powerful set of menus spread over several tabs. There’s a ton of power there just in selecting options. ZAP has far fewer configurable options in the Fuzzer pane, but supports custom scripts for payload processing. One nice side-effect of the scripting is that it would solve one of my longtime complaints about Intruder - I have to recreate rules in the configuration far too regularly. Overall, I think I still have a clear preference for Burp’s fuzzing, but I think if ZAP’s scripting API was well-documented with high-quality editing tools built-in, it could easily be the better of the two for power-users.

Proxying Mobile Apps

If you’ve been working through the series, you might have noticed we didn’t touch on mobile apps. I’ve used Burp for testing mobile apps many times. It can certainly have frustrating moments, but ZAP has found ways to exacerbate that. For example, in Burp, once your device is proxying, you can open http://burp and grab the CA cert so that you can set up the certificate trust. ZAP prevents that, making you side-load your CA cert because the web API interface to retrieve it isn’t available when you’re not accessing it over local loopback. This is done as a security measure, but I can think of several other ways of securing it. I don’t recommend using ZAP for mobile apps.  Burp is my choice for mobile apps of the two.

Update: A clarification and a correction here based on Simon's feedback. Setting ZAP to listen on a different network interface wasn't a problem for me. I missed the setting that would enable the API on my lab network interface, and in troubleshooting it I read something that I interpreted as a statement that it was a security decision to not serve the CA cert outside local loopback. If you have a secure WLAN for testing, you can enable it, and Simon's post identifies where the setting is. With that pain point solved, I don't think I have any major reservations about using ZAP for mobile. I didn't end up completing a good test of its capabilities for mobile testing, so I can't really recommend it, but given the information I have, it should be as capable as Burp.

Automated Tools for Spidering and Scanning

There’s a lot of overlap between the two in terms of their spidering and scanning offerings. I had some strong criticisms for ZAP in the posts related to this topic, which might suggest that Burp is better. However, it’s really not that clean-cut. The areas where ZAP struggled the most (e.g. handling complex auth flows), I have found the Burp struggles with as well. I’ve worked several times with a client that has a homegrown OAuth IDp as part of their app suite, and Burp is just completely unable to successfully authenticate automatically.

Overall, I think Burp has a few advantages here, a big one being Collaborator, which is a hosted service (you can self-host or use Portswigger’s instance) that generates a unique hostname and then reports back to you when it sees service interactions such as DNS lookups and HTTP traffic. This is something that I think could absolutely be added to ZAP, and may already exist in a third-party plugin. Update: As Simon identified in his response post, the OAST Add-On handles the out-of-band capabilities for ZAP. He included references to additional information on it, so check out what he had to say. I didn't look at it for this series, but I look forward to trying OAST out and writing about it further in the new year.

Scripting and Customization

ZAP seems to have a very robust scripting interface with all sorts of capabilities, but it’s not approachable and barely documented. The tools within ZAP can do some of it for you, if you use the ZEST scripting language, but a drawback is that if you use the GUI-based options to start a script, it’s not editable manually. The code editor built into ZAP didn’t seem to provide any completion suggestions (correction: while I didn't note any completion in my use of the scripting interface, the there's completion for parameters - which is the most crucial place for it, in my opinion)  or syntax checking either, making it a bit rough to work on complex things. 

On the other hand, Burp doesn’t really have much of a scripting interface (some extensions provide one though). Both apps have systems for building extensions or plugins with Java, which I can’t really compare because I haven’t built extensions for either one. So ZAP is the clear winner here for me, but I’m still not a fan, yet. I really want an MDN-equivalent for its scripting APIs, and either a better built-in editor or a language plugin for VSCode. But I also recognize that’s a heck of a tall ask for a volunteer-driven project.

In Summary…

I didn’t entirely know what to expect when I started this series. I’d heard that ZAP is quite good, probably pretty close to Burp, but different. I worried the interface wouldn’t be intuitive for me, but most of the time it was. I’d seen some fanfare around the HUD a couple of years ago, and was excited about that. I’d also seen a lot of the documentation around ZAP that seemed to prioritize its automated scanning capabilities, so expected them to be the main focus for it. What I found was essentially the opposite of what I expected. The HUD is a neat idea, but I didn’t really like using it. The real fundamental pieces, the proxying, setting scope, repeating requests, and fuzzing really impressed me. I think it’s a great tool, and I think the maintainers have done an excellent job with it. I wouldn’t hesitate to reach for ZAP for manual testing instead of Burp, and I’ll likely use its ATTACK mode as a secondary check as well. I do think that if I were using it as my daily driver, I would need to become proficient with its scripting tools to get the most out of it.

Day 12 will finish out the series with some defense-focused Content-Security-Policy action.