Introduction
Honeypots are computers that masquerade as vulnerable systems to attract potential attackers. By mimicking weak or outdated software, honeypots entice attackers looking for an easy target. This provides security professionals with insight into attackers’ tactics, techniques, and procedures (TTPs), helping them understand and mitigate potential threats more effectively. In addition, honeypots are a fundamental detection and intelligence-gathering tool for many threat intelligence providers.
In this blog series we will detail how to evade honeypots using JA3 hash randomization to enumerate and analyze the honeynets of threat intelligence providers. These providers frequently supply their intelligence to different security tools and services, like Web Application Firewalls (WAFs) and Content Delivery Networks (CDNs). By accurately mapping and excluding their honeynets, scans can be conducted more discreetly, making detection considerably more challenging. While honeynets are a valuable tool and an important part of threat feed intelligence, it’s good to remember that savvy attackers can evade these kinds of routine detection mechanisms and organizations should seek to include security measures that focus on application/API usage and behavior patterns like the Ghost platform.
In part 1 of this blog we’ll define and describe the JA3 hash and discuss the tools we used and how to randomize the JA3 hash.
In part 2 we’ll show how to enumerate a honeynet and answer the following questions:
- What are the IP’s of the honeypots so we can avoid them?
- Who hosts the honeypots? In what cloud services are the honeypots deployed, and thus what networks can they effectively protect?
- Can we easily and accurately fingerprint the honeypots?
- Can we build a Censys/Shodan search string using these fingerprints?
In part 3 we’ll describe how to analyze a honeynet and present some of our findings including:
- The specific vulnerabilities flagged by the honeypots (the CVEs they can “see”)
- The vulnerability categories prioritized by the honeynet
- The services the honeypots mimic
JA3 Hash
The JA3 hash is a fingerprinting technique used in network security to identify and classify the cryptographic properties of a TLS (Transport Layer Security) handshake. It has been a staple for incident response teams and threat intelligence platforms since its fingerprinting technique was published by Salesforce in 2017. The hash allows for quick identification of hacking tools like Cobalt Strike, Sliver beacons, and SQLmap, and is reliable over time because threat actors often reuse such tools without modification.
When a client and server establish a secure connection over TLS, they exchange a series of messages called the handshake. The JA3 hash is generated by analyzing the specific characteristics of this handshake, such as the TLS version, supported cipher suites, and extensions used. The combination of these attributes is unique to the cryptographic configuration of each TLS client or server.
The TLS handshake
The JA3 hash is valuable in network security monitoring. By comparing the JA3 hash of a network connection against known hashes associated with malicious activity, security professionals can identify potentially malicious TLS connections, and even what specific tools are being employed by the threat actor.
Since the JA3 hash is generated purely from the handshake parameters and does not involve decrypting the actual TLS traffic, it provides a useful tool for identifying and flagging suspicious TLS connections without violating the privacy and confidentiality of the communication.
Fingerprinting Countermeasures
For better coverage of internet-wide reconnaissance, like scanning for devices with specific vulnerabilities (as Shodan and Censys do regularly), it’s necessary to bypass the protection of cloud-based Web Application Firewalls (WAFs) and Content Delivery Networks (CDNs) and to avoid honeynets deployed randomly throughout the internet by threat intelligence providers and other research institutions.
Scans performed without sufficient fingerprinting countermeasures will be blocked by WAFs and CDNs or detected by honeynets and end up on threat intelligence feeds. This will result in scans being blind to significant portions of cloud infrastructure where those feeds are a source of Indicators of Compromise (IOCs). A typical way to mask scan traffic is to use a rotating proxy service to randomize the source IP (with high reputation IPs) and the User-Agent header in each request. This by itself is enough to bypass most WAFs, but not to scan targets behind cloud-based WAFs or CDNs like CloudFlare. Scanning targets behind those technologies requires more advanced techniques, such as cloning the header order for a common browser like Chrome, mimicking User-Agent strings, or cloning or randomizing the JA3 hash of the browser configuration being spoofed.
While these countermeasures can be successful at hiding scans from threat intelligence honeynets, they aren’t guaranteed to work. The only guarantee of evading detection by honeynets is to avoid scanning them, but this requires knowing the IP address of each honeypot.
Our approach to determining the IP addresses of a threat intelligence company’s honeypots is founded in JA3 hash randomization. Since AWS is a likely place to deploy a honeypot, we started with aggressive scans against all AWS’ published IPv4 prefixes. We rotated the JA3 hash for each request so it was unique and used with only one target IP address. We then searched for the IP of our scanner box using the webapp and API for one of our target feeds and found that over 200 of our JA3 hashes had been recorded. By correlating those JA3 hashes back to our scan targets we were able to eventually confirm the IPs of 200+ honeypots. At this point we could have followed the same process for other large IP blocks, but we opted for an optimized approach of using additional fingerprints from the confirmed honeypots to generate smaller target IP blocks. We will provide more details on this process in part 2 of this blog.
JA3 Randomization
Most reconnaissance tools have an option to customize HTTP headers. Changing the headers can help users evade detection techniques like User-Agent string analysis and header order analysis. What these tools don’t provide is a way to avoid JA3 detection. This requires a feature that allows users to customize or randomize the TLS attributes that are used in calculating the JA3 hash.
When we started this research we intended to use nuclei, Project Discovery’s vulnerability scanner, for our scanning. However, we learned that its JA3 hash was the same across most builds. We found this was also true for BurpSuite****************and httpx, another Project Discovery tool that shares common libraries with nuclei. This makes it easy to detect and block these tools using their JA3 hashes. We’ve noticed this happening more often with BurpSuite.
We submitted a feature request to Project Discovery to include a new flag (-tlsi, -tls-impersonate) in nuclei and httpx for randomizing the JA3 hash, and this feature is now part of the latest build. While this does randomize the JA3 hash, nuclei does not include the hashes in any of its logs making it difficult to correlate the hashes with the target IP addresses.
Looking for an alternative that would log the hashes and target IP’s, we created a setup using an nginx proxy to capture the requests and an nginx server running on the same machine to capture the responses. Using the proxy, we performed tests with BurpSuite , nuclei, and httpx to see what impact the proxy and various flags has on their JA3 hashes. We learned that although we were able to proxy nuclei and httpx (using the -proxy flag) and to log the hashes and IP’s, the JA3 hash randomization doesn’t work correctly when used in conjunction with the -proxy flag. The findings from our testing are described in the BurpSuite and httpx/nuclei sections below. Although this wasn’t our final setup, we’ve included our nginx build script and configuration files at the end of the blog.
JA3 randomization can be done with Python's aiohttp and requests libraries (we’ve included two example python scripts at the end of the blog), but in the end we decided to scan with httpx and nuclei, use tcpdump to capture the traffic during scanning, and then process the resulting pcap files with ja3.py, a JA3 python script from SalesForce. One advantage of the nginx proxy and python options is that you can map the JA3 hashes and target IP addresses in realtime, whereas the ja3.py script is run offline after scanning is complete.
BurpSuite
We sent requests from BurpSuite’s built in Chromium browser through our nginx proxy. The json below shows a portion of the metadata for one of those requests. Depending on configuration options, Burp produces one of two JA3 hashes:62f6a6727fda5a1104d5b147cd82e520 or 8bd06f4341a65d44a68bd2cef7cbedc6. In fact, if you proxy a tool through BurpSuite, such as proxying python’s requests library through Burp, you will assume BurpSuites’s JA3 hash shown below. Organizations are increasingly tracking these JA3 hashes which is why we’ve seen more websites blocking Burp.
httpx/nuclei
nuclei and httpx are both built on the golang net/http library which uses crypto/tls. We wanted to see whether nuclei, httpx, and a basic net/http golang program share the same JA3 hashes. In order to make this comparison we wrote a basic golang net/http program and then issued requests from all three programs through our nginx proxy.
The golang program included at the end of the blog issues an http request using net/http. When run, it yields a JA3 hash of 3fed133de60c35724739b913924b6c24, which we’ve concluded is the default hash when using the standard values for the net/http library and, as we’ll show below, is a different hash than the ones shared by nuclei and httpx.
When httpx is run without the -tlsi flag it yields the same JA3 hash for both requests as shown below. The first request is the proxy/TLS connection which tells the proxy to initiate a connection to our proxy. The second request is the one forwarded by the proxy once the connection has been established. Note the JA3 hash is the same and the User-Agent has been randomized in the second request.
When httpx is run with the -tlsi flag the JA3 hash is randomized as expected for the proxy CONNECT request, but the second request has the same default hash we saw in the last test.
When we run httpx with the -tlsi flag, but without the -proxy flag, the request has a random JA3 hash, showing that JA3 randomization does work, but not with the -proxy flag. We confirmed this theory using a different proxy service (OxyLabs). We again saw the same default hash 473cd7cb9faa642487833865d516e578 which confirms the JA3 is randomized only to the proxy, and subsequent requests use the standard nuclei/httpx JA3 hash.
JA3 randomization is also affected by the -unsafe flag which allows httpx to make intentionally broken HTTP requests such as the following:
GET / HTTP/1.1\r\nHost:\r\nGET /metadata HTTP/1.1\r\nHost: 169.254.169.254
This is an example of request smuggling; a single request that actually contains two requests. When -unsafe is specified, both nuclei and httpx forward the request as-is allowing the user to potentially exploit a vulnerability in the receiver. Without -unsafe, both nuclei and httpx strip everything after the first Host header.
The output below shows the impact of the -unsafe flag on the JA3 hash for the same httpx command, with and without -proxy.
Notice that both requests have the same JA3 hash: 19e29534fd49dd27d09234e639c4057e. When -unsafe is used, httpxand nuclei ignore -tlsi and use a static JA3 hash. Depending on the scan configuration, that hash will be either 19e29534fd49dd27d09234e639c4057e or 473cd7cb9faa642487833865d516e578. As an aside, it appears that the -proxyflag interferes with the -unsafe flag. When -unsafe is supplied the -path value should be forwarded as-is, but the output above shows that when -proxy is used, -unsafe is ignored and the path is truncated anyway.
Conclusion
In part 1 of this series we’ve defined the JA3 hash, discussed its usefulness for detecting malicious tools and evading honeypot detection, and given examples of tools and configurations that can be used for JA3 randomization. Although we’ve focused on randomizing JA3 hashes, there is significant value in being able to specify custom JA3 hashes. This would allow pentesters and attackers to impersonate the JA3 hash of common tools or browsers and decrease the likelihood of being detected. For example, we asked multiple iPhone owners, all with different hardware, software patch levels and so on, to make a request to our proxy. As we expected, the JA3 was the same across all devices. This would be a good impersonation candidate.
| Tool | JA3 hashes observed | Notes |
| nuclei | 473cd7cb9faa642487833865d516e578 19e29534fd49dd27d09234e639c4057e | The “19e” hash value is observed only when sending ‘unsafe’ raw requests. |
| httpx | 473cd7cb9faa642487833865d516e578 19e29534fd49dd27d09234e639c4057e | The “19e” hash value is observed only when sending ‘unsafe’ raw requests. |
| golang net/http | 3fed133de60c35724739b913924b6c24 | |
| BurpSuite | 8bd06f4341a65d44a68bd2cef7cbedc6 62f6a6727fda5a1104d5b147cd82e520 | The 62f hash value was observed when performing the first request to a page, all subsequent requests were the 8bd value. If you proxy a tool through Burp, it will always have the 8bd value. |
| python requests | 6776eeb3122863bfadbffae2afb4dd8d | |
| python aiohttp | 047bfa6321a7921f57d5b91d04dcedaf |
Ready for more? Continue on to Part 2 now: An Attacker's Guide to Evading Honeypots Part 2
