How to reproduce the results (Section 5.2) on the paper?
We wanted to measure the current state of SPF validation by SMTP servers in the wild with respect to lookup limits. In our experiment, we focus on the DNS lookup limits specified in the standard; namely: (1) total DNS lookup and (2) void lookup. Please refer to Section 5.1 of our paper to understand the methodology of our scanning.
-
For this experiment, there are two datasets ; 1) hashed value of the scanned recipient MTAs, 2) corresponding DNS logs made by the MTAs due to our scanning. To evaluate the consistency of our paper, one should: (1) download our datasets, (2) run our analysis scripts.
Reproducing the analytics from the datasets
This section introduces a way to verify the results produced via our Internet wide scanning.
Datasets and scripts
(1) Datasets and prerequisites for the analysis.
| Filename | Download | Description |
|---|---|---|
List of scanned MTAs (hashed) |
link | |
DNS Logs |
link |
(2) Scripts for the analysis
| Filename | Download | Description |
|---|---|---|
generate-fig7-data.py |
link | This script is for analyzing the BIND logs associated with the scan and generating data for reproducing Figure 7. |
How to run the scripts?
- Requirements: You'll need ~2GB of storage and python3 installed in your machine.
- Create a directory named spf-usenix-fall-2024-artifacts anywhere you want.
- Download the specified json and zip files in this directory.
- Unzip the bind-logs.zip file within this directory. If done correctly, there should be a bind-logs directory under the base spf-usenix-fall-2024-artifacts directory.
- Navigate to the spf-usenix-fall-2024 directory in a terminal.
- Run python3 generate-fig7-data.py
- If everything goes smoothly up until this point, you should see the numbers and their explanations. Please match it with Section 5.2 of our paper.