Identifying cases of DNS misconfiguration: Not quite censorship

Leonid Evdokimov, Vasilis Ververis 2017-11-24

Country: Brazil, worldwide

OONI tests: Web Connectivity, HTTP requests

Probed ISPs: AS1916 (Associação Rede Nacional de Ensino e Pesquisa), AS262650 (Kyatera Informatica Ltda), AS2715 (Fundacao de Amparo a Pesquisa/RJ), AS27699 (TELEFÔNICA BRASIL S.A), AS28573 (CLARO S.A.), AS52873 (SOFTDADOS CONECTIVIDADE), AS7738 (Telemar Norte Leste S.A.), AS8167 (Brasil Telecom S/A - Filial Distrito Federal)

Measurement period: July 2016 — November 2017

Website inaccessibility reasons: IPv6 or DNS misconfiguration

We recently noticed that pernambuco.com, a regional news portal, has not been reachable from several networks in Brazil for quite some time. By looking at OONI data, we found that the site was not reachable and that it presented signs of DNS-based blocking due to empty DNS responses that is usually a symptom of potential internet censorship. But upon further analysis, we found a number of DNS misconfigurations of the nameservers hosting the domain in question. A nameserver is a function of a DNS server that implements a network service for providing responses to queries against a directory service. In this article we present an analysis of possible DNS misconfigurations that may have caused pernambuco.com to be inaccessible. We also provide some solutions to resolve DNS misconfiguration for pernambuco.com and other sites potentially affected in Brazil and worldwide.

OONI web_connectivity tests

OONI tries hard to apply Hanlon’s razor to every statement about network interference: never attribute to censorship that which is adequately explained by misconfiguration. Thanks to Lucas from CodingRights who gave us access to his networks, the numerous OONI measurements submitted by volunteers and RIPE Atlas measurements, we were able to investigate the inaccessibility of pernambuco.com in Brazil.

DNS resolution failures

The basic issue that we identified is that the recursive DNS nameserver provided by Virtua ISP failed to resolve the domain, responding with SERVFAIL (Server failure) meaning that “the name server was unable to process this query due to a problem with the name server” according to RFC1035.

Below we include the output of the DNS lookup utility dig querying the domain name pernambuco.com:

One of the ways to check if the domain is blocked by some local DNS policy is to attempt to resolve it directly through an authoritative nameserver. If DNS name resolution works that way, it suggests that the case could be some misconfiguration and not a case of DNS-based filtering. And pernambuco.com passed the test:

While checking various resolution paths we discovered that ns2.upx.com.br, resolving to the IPv6 address 2001:19f0:ac01:b3:5400:ff:fe46:4676, was not responding to DNS queries. The first assumption we made was that the recursive DNS nameserver has a strong preference for IPv6 over IPv4 and fails fast if the IPv6 endpoint is not reachable. However, the resource ns1.upx.com.br had an IPv6 address, so we assumed that there is something special about the resource ns2.upx.com.br and found that it was the only authoritative nameserver with an IPv6 glue record in the top-level domain zone.

Through RIPE Atlas measurements, we verified that ns2.upx.com.br works without errors through IPv4 via DNS/TCP, but is broken through IPv6 both via DNS/UDP and DNS/TCP.

We got in touch with UPX Technologies, the service provider of the aforementioned DNS nameserver, and reported the issue. The technical support team was very collaborative and quickly resolved the issue of IPv6 connectivity for DNS/TCP and DNS/UDP. We appreciate the amount of cooperation and support that we received from the UPX team, even though we are not their customers.

Unfortunately though, the recursive DNS nameserver still responds with the SERVFAIL error code and the issue affects many Brazilian networks, but not all of them. This strongly suggests that it may be a misconfiguration issue, rather than a network filtering policy. In addition to Brazilian networks, this issue also appears to be affecting a number of other networks worldwide and the failure was cached according to consequent resolution latency.

RIPE Atlas measurement showing worldwide failures

Other affected domains include aquipe.com.br, assineodiario.com.br, brunoprado.com.br, clubediario.com.br, espacodaprevidencia.com.br, radiogloborecife.com.br, tvonorte.com.br and wakeworld.com.br. All of these domains present the same failure and symptoms. But some other domains “hosted” by UPX Technologies were not affected, such as dpnet.com.br and the resource upx.com.br.

Most of the affected domains listed above look like stale, “dead” websites, while only two of them seem “active”. What is common amongst the failing domains (SERVFAIL response code) is the difference between the NS records at the top-level domain nameserver and the authoritative nameserver that was clearly visible in the trace of the delegation path from the root name servers; dig +trace.

NS records pointing to a CNAME is a bad practice

The NS records of the affected domains are americadonorte.upx.com.br and americalatina.upx.com.br. These records are CNAME records in the upx.com.br zone and according to RFC’s 1912 common DNS errors section (dating back around 11 years ago): “Having NS records pointing to a CNAME is bad and may conflict badly with current BIND servers. In fact, current BIND implementations will ignore such records, possibly leading to a lame delegation”.

Below we include a proof of concept to verify this issue in BIND, the most widely used DNS server:

Workaround

We propose the following possible solutions to fix this issue:

Responsible disclosure

On 15th November 2017 we reported the issue to UPX. As a result, they quickly resolved the IPv6 issue. They also explained that the CNAME incident was a fix for some operational issues and that they were aware that this could potentially break backward compatibility. UPX assured us that the affected customers were notified about the migration to new NS names a long time ago.

UPX also advised us to notify the administrators of the affected websites, pointing them to UPX customer service to avoid potential trust issues. We notified the administrators of the affected “active” websites on 23rd November 2017, but some of these email notifications bounced back.

¯\_(ツ)_/¯

We believe that the issue is indeed a misconfiguration and not an intentional, network filtering policy. We hope that this report will help others to troubleshoot and fix DNS misconfiguration affecting the accessibility of sites around the world.