(NOTE: The Nebulous API was retired on 4/15/2020. The command line tool will remain available on GitHub. Thank you to everyone who made use of NebulousAD and sent us valuable feedback!)
Last month we released NebulousAD, a free tool for checking user credentials in Active Directory against more than 2.5 billion unique passwords that have been compromised in data breaches.
The tool was made with system administrators in mind. It’s available as a precompiled executable or can be built from source, it doesn’t require you to download and host terabytes of data, and it was designed to work well with SIEM tools and Windows Task Scheduler. You can watch Robert, our Director of R&D, present NebulousAD at the BSides Las Vegas security event here.
Today we’re happy to announce an update to the NebulousAD tool that brings two important changes: improved documentation for the tool and API that will make getting up and running even easier, and support for the privacy-preserving k-anonymity model of checking password hashes.
We’re particularly excited about the k-anon support since privacy is at the core of what we do. Although we don’t log or store the hashes that are sent to the NebulousAD API, we’d much rather not even be able to if we tried. This is what k-anon enables.
We took inspiration from Troy Hunt’s Pwned Passwords service, which has been using k-anon to check breached passwords since early 2018. The model, which is explained in detail by Junade Ali at Cloudflare, is simple in all the best ways.
Rather than sending an entire hash to our API to check if that exact hash shows up in our database, with k-anon only the first few characters of the hash are sent. This “hash prefix” will match not just one, but a set of hashes in our database that all happen to start with those characters (but then differ in their subsequent characters, since the hashes are all unique in full). The entire set of hashes from the database with a matching prefix are then sent to the client, where an offline check is performed to determine if the original hash matches any in the returned set.
Because we never receive the full hash, we don’t know if any of the hashes in the prefix-match set are actually the same as the one being audited by the client. And if there is a match (which we wouldn’t know), we also wouldn’t know which of the hundreds of hashes in the set it is.
NebulousAD uses a hash prefix length of five characters, and returns a minimum of 727 and maximum of 1,004 results for any valid five-character prefix. The pseudo-random nature of hashes makes it so that while hashes are distributed relatively evenly across all possible prefixes, there are variations which result in that range of response size.
As an example, the text “ilovemydog” has the SHA-2(NTLM) hash of:
The first five bolded characters “be266” are the prefix. This is what would be sent from the client to our API. Querying this prefix against the Nebulous database results in 856 matches (this number may change as we add more data in the future). As mentioned above, we wouldn’t know which, if any, of the 856 hashes with the be266 prefix match the original full hash (spoiler: in this case “ilovemydog” is definitely in Nebulous). All 856 matching hashes will be returned to the client, where the final check for a full match is made.
Using the k-anon model makes it significantly harder for us, or anyone who may be able to intercept your traffic, from gaining any useful information from monitoring your interactions with the Nebulous API.
You can get started using NebulousAD by heading over to our GitHub page. As always, we welcome and encourage any questions or feedback at firstname.lastname@example.org. If you have had a chance to try the tool, we would also appreciate if you could take a couple minutes to fill out this feedback form.