Background

Introduction

On this page, we provide concise technical background information about our measurement. This information is targeted at DNS operators and academic researchers.

Measurement Goal

The goal of the OpenINTEL measurement platform is to capture daily snapshots of the state of large parts of the global Domain Name System. Because the DNS plays a key role in almost all Internet services, recording this information allows us to track changes on the Internet, and thus its evolution, over longer periods of time. By performing active measurements, rather than passively collecting DNS data, we build consistent and reliable time series of the state of the DNS.

What do we measure?

OpenINTEL performs two types of measurements: forward and reverse DNS measurements.

Forward DNS measurements

For forward measurements, the measurement system uses a fixed set of DNS queries, which it sends once every 24 hours for every second-level domain in a TLD. This set of queries currently consists of the following queries:

  • SOA
  • NS (3)
  • A (1)
  • AAAA (1)
  • MX (3,4)
  • TXT
  • DNSKEY
  • DS
  • NSEC3 (2)
  • CAA
  • CDS (5)
  • CDNSKEY (5)

  1. These queries are also sent for the www label (e.g. www.example.com).
  2. We send a query for a non-existent domain name to record authenticated denial-of-existence data, but only for DNSSEC-signed domains.
  3. We resolve the A and AAAA records for these records in a separate infrastructure measurement.
  4. We resolve associated TLSA records for ports 25, 465 and 587 in a separate infrastructure measurement.
  5. We only resolve these records for DNSSEC-signed domains for which at least a DNSKEY or DS record exists.

All response records, including full CNAME expansions and RRSIG signature records are stored.

Reverse DNS measurements

For reverse DNS measurements, we measure the delegation structure of the reverse address space (SOA and NS queries). Once we know the delegation structure, we query individual PTR records for parts of the name space that we know to be delegated. The measurement is randomised over the address space to prevent high outgoing query loads to specific name servers that are authoritative for significant delegations (e.g. at /16 or larger level). We currently only measure the reverse DNS address space for the IPv4 Internet.

What can DNS operators expect?

Forward DNS measurements

As a DNS operator, you can expect that our measurement system will send you 11 queries per day per domain you operate, and in case the domain is DNSSEC-signed, 12 queries. The system has been designed to distribute measurements over time, to reduce the impact the measurement has on busy name servers that are authoritative for large numbers of domains. We regularly inspect flow information about our measurements to monitor query rates. In general, only if you operate a very busy name server in terms of numbers of domains, should our measurement be visible in statistics. If you are a DNS operator and you feel our measurement is impacting your infrastructure, please read on.

Reverse DNS measurements

You can expect a single PTR query for every IP address for which you manage the reverse DNS. You may also see sporadic SOA and NS queries for names at delegation points.

What should I do if the measurement impacts my infrastructure?

Please go to our Problems page for information on how to contact us and how you can block our traffic if necessary.

What will the data be used for?

The data collected by the OpenINTEL measurement platform will only be used for academic research and may not be used for commercial purposes. Read more about getting access to data for research on our Data Access page.

Where can I find more detailed information?

Our data dictionary explains the schema of the OpenINTEL data files in Apache Avro format. Use this information to process files that you can download from our data access page.

If you would like to learn more about how OpenINTEL works, please consult one of the references below. These academic papers are devoted to the design and use of OpenINTEL.

  • A High-Performance, Scalable Infrastructure for Active DNS Measurements

    In IEEE Journal of Selected Areas in Communications, Volume 34, Issue 7

    Read a pre-print of the paper

  • The Internet of Names: a DNS Big Dataset

    Actively Measuring 50% of the Entire DNS Name Space, Every Day

    In Proceedings of ACM SIGCOMM 2015, August 17-21 2015, London, UK

    Read the abstract

    View the poster