DNS unblocking using Dnsmasq and HAProxy

As I mentioned in my previous post, the open source DNS forwarder Dnsmasq is ideal for the DNS part of DNS unblocking. I’m running Dnsmasq on a $30 Raspberry Pi credit card sized mini computer which is up 24/7 anyway since it also handles all VOIP phone calls at home. I point my Mac, Apple TV and iPad to the RPi as the primary DNS server.
On the server side, I’ve setup a HAProxy instance using just a single IP address as a proof of concept. This poor-man’s approach works beautifully with SNI-capable devices like my Mac and iOS devices. I think newer Android devices are SNI-compatible as well but I haven’t tested it. Windows 7 and up should be OK too. Older devices like the Playstation 3 or Xbox 360 are most likely not SNI-compatible and won’t work with my highly cost-efficient single IP address approach. Unfortunately, even some of the newest multimedia players don’t support SNI.

The HAProxy server is running on a lowend virtual private server in the U.S. As a starting point, feel free to use my proof of concept server as shown in the Dnsmasq configuration below. In the web browser, you should be able to watch Netflix, Hulu/HuluPlus, free episodes/TV shows on MTV, Disney XD, Syfy, NBC, ABC, Vevo, Crackle, PBS and CWTV. Netflix works on iPad and Apple TV too. HuluPlus could work on iOS as well.

Please remember, even though I’m planning to keep the HAProxy server up for some time to come, this is just a proof of concept and not a fully fledged DNS unblocking service.

On Debian-based Linux distributions, add the content below to a file named dnsmasq-catchall.conf in /etc/dnsmasq.d and it will get included by Dnsmasq. If Dnsmasq is running, i.e. on 192.168.178.99, you can test it using:

dig @192.168.178.99 trick77.com

This should bring up a few NS and A records for this site.

dig @192.168.178.99 abc.go.com

The result should be an A record for abc.go.com pointing to 199.204.184.146

/etc/dnsmasq.d/dnsmasq-catchall.conf:

address=/abc.go.com/199.204.184.146
address=/api.watchdisneyxd.go.com/199.204.184.146
address=/api.watchabc.go.com/199.204.184.146
address=/release.theplatform.com/199.204.184.146
address=/www.crackle.com/199.204.184.146
address=/api.crackle.com/199.204.184.146
address=/ios-api.crackle.com/199.204.184.146
address=/appletv.crackle.com/199.204.184.146
address=/ios-api-us.crackle.com/199.204.184.146
address=/ios-api.crackle.com/199.204.184.146
address=/android-api-us.crackle.com/199.204.184.146
address=/xboxone-api-us.crackle.com/199.204.184.146
address=/ps3-api-us.crackle.com/199.204.184.146
address=/roku-api.crackle.com/199.204.184.146
address=/content.uplynk.com/199.204.184.146
address=/content-us-east-1.uplynk.com/199.204.184.146
address=/www.crunchyroll.com/199.204.184.146
address=/api.crunchyroll.com/199.204.184.146
address=/static.discoverymedia.com/199.204.184.146
address=/www.dramafever.com/199.204.184.146
address=/token.dramafever.com/199.204.184.146
address=/link.theplatform.com/199.204.184.146
address=/s.hulu.com/199.204.184.146
address=/play.hulu.com/199.204.184.146
address=/www.iheart.com/199.204.184.146
address=/www.last.fm/199.204.184.146
address=/ws.audioscrobbler.com/199.204.184.146
address=/ext.last.fm/199.204.184.146
address=/www.logotv.com/199.204.184.146
address=/activity.flux.com/199.204.184.146
address=/j.maxmind.com/199.204.184.146
address=/mog.com/199.204.184.146
address=/www.mtv.com/199.204.184.146
address=/c.brightcove.com/199.204.184.146
address=/video.nbcuni.com/199.204.184.146
address=/video.nbc.com/199.204.184.146
address=/video.syfy.com/199.204.184.146
address=/signup.netflix.com/199.204.184.146
address=/www.netflix.com/199.204.184.146
address=/appboot.netflix.com/199.204.184.146
address=/cbp-us.nccp.netflix.com/199.204.184.146
address=/a248.e.akamai.net/199.204.184.146
address=/api-global.netflix.com/199.204.184.146
address=/movies.netflix.com/199.204.184.146
address=/movies1.netflix.com/199.204.184.146
address=/secure.netflix.com/199.204.184.146
address=/moviecontrol.netflix.com/199.204.184.146
address=/api.netflix.com/199.204.184.146
address=/api-us.netflix.com/199.204.184.146
address=/uiboot.netflix.com/199.204.184.146
address=/cbp.nccp.netflix.com/199.204.184.146
address=/ios.nccp.netflix.com/199.204.184.146
address=/xbox.nccp.netflix.com/199.204.184.146
address=/nccp-nrdp-31.cloud.netflix.net/199.204.184.146
address=/nintendo.nccp.netflix.com/199.204.184.146
address=/playstation.nccp.netflix.com/199.204.184.146
address=/nrdp.nccp.netflix.com/199.204.184.146
address=/android.nccp.netflix.com/199.204.184.146
address=/www.pandora.com/199.204.184.146
address=/mediaserver-sv5-rt-1.pandora.com/199.204.184.146
address=/tuner.pandora.com/199.204.184.146
address=/urs.pbs.org/199.204.184.146
address=/video.dl.playstation.net/199.204.184.146
address=/api.wipmania.com/199.204.184.146
address=/www.rdio.com/199.204.184.146
address=/www.smithsonianchannel.com/199.204.184.146
address=/once.unicornmedia.com/199.204.184.146
address=/media.mtvnservices.com/199.204.184.146
address=/www.spike.com/199.204.184.146
address=/udat.mtvnservices.com/199.204.184.146
address=/www.thewb.com/199.204.184.146
address=/www.cwtv.com/199.204.184.146
address=/media.cwtv.com/199.204.184.146
address=/pdl.warnerbros.com/199.204.184.146
address=/cdn.wwtv.warnerbros.com/199.204.184.146
address=/www.vevo.com/199.204.184.146
address=/sb.vevo.com/199.204.184.146
address=/videoplayer.vevo.com/199.204.184.146
address=/www.vh1.com/199.204.184.146
address=/screen.yahoo.com/199.204.184.146
address=/geo.yahoo.com/199.204.184.146
address=/mvid.yql.yahoo.com/199.204.184.146
address=/hls.video.query.yahoo.com/199.204.184.146
server=/beta.abc.go.com/8.8.8.8
server=/cdn.media.abc.go.com/8.8.8.8
server=/site.abc.go.com/8.8.8.8
server=/cdn.abc.go.com/8.8.8.8
server=/cdn.media.abc.go.com/8.8.8.8
server=/a.verdict.abc.go.com/8.8.8.8
server=/theview.abc.go.com/8.8.8.8
server=/adsatt.abc.go.com/8.8.8.8
server=/ll.static.abc.go.com/8.8.8.8
server=/static.east.abc.go.com/8.8.8.8
server=/share.mog.com/8.8.8.8
server=/logger.mog.com/8.8.8.8
server=/search.mog.com/8.8.8.8
server=/support.mog.com/8.8.8.8
server=/www.mog.com/8.8.8.8
server=/api.mog.com/8.8.8.8
server=/api.au.mog.com/8.8.8.8
server=/images0.mog.com/8.8.8.8
server=/images1.mog.com/8.8.8.8
server=/images2.mog.com/8.8.8.8
server=/images3.mog.com/8.8.8.8
server=/cdn2.mog.com/8.8.8.8

Once HAProxy is up and running, it will show a lot of statistics if you’ve enabled the built-in web interface:
haproxy-web-ui

Please let me know in the comments below once you have successfully set up your own DNS unblocking solution!

36 thoughts on “DNS unblocking using Dnsmasq and HAProxy

  1. Can someone point me to an article that allows me to do this using DNSMasq and SNI Proxy instead of HAProxy?

  2. If “Ed” is reading this: please resend me your email address using the contact form, the address got lost somehow in the contact form sent to me.

    Cheers
    Jan

  3. My setup is dnsmasq and haproxy running on VPS, and just setting my DNS of the media PC to the VPS since I get 20ms ping times and I don’t use my HomeServer for general browsing.

    Has anyone scripted anything to make it quick and easy to add more domains to the haproxy and dnsmasq files? I like plink (from putty) to automate stuff with very hacky and basic windows .bat files. Stuff like appending stuff to a file is cake, but adding stuff in the middle of files (haproxy conf) is out of my league. Something would need to sed or keep track of the last rule for each section?

    Thanks very much for putting together these informative posts!

  4. got everything working right, have the non-sni based setup
    all the sites work as they should, including xfinityTV, however the liveTV section of xfinitytv isnt working, im guessing they will need to be added to config.json

    if anyone can give me the the few lines to add i would appreciate the help

  5. This is great. I now have a cheaper solution for Netflix (which is 99% of my use).
    I also like to be able to occasionally rent some new release movies when they are not yet available in Australia. Vudu doesn’t seem to work with this setup. I did wireshark the transaction using unblock-us to see what goes through and what doesn’t. I seemed to be able to get to the watching point and got a black screen. Is there another service that I can use that allows this (preferrably available on AppleTV or PS3) that I can use this setup for?

  6. Jan- thank you for writing this up — but it falls short of my goal, which is your anti-goal. I want to route *all* netflix traffic through this and not just the geofenced features.

    Why? Verizon/Comcast are intentionally letting Netflix underperform and routing traffic through a VPN/remote server overcomes the problem. So instead of logging into a VPN, I’d love to route my traffic through a remote VPS.

    Could you help me identify the correct configuration to route all netflix.com traffic to the VPS?

  7. Hello,
    Thanks for this invention :)
    I have the same question as fred, how to do the root level unblocking, I believe haproxy wont be able to do that, right?
    BR, Sherif.

  8. Is it true that sni pure works with ios devices? I am trying to run pandora and it works in the browser, but in my iphone it says “Cannot connect to Pandora”; I tried my vps server configuration and yours (199.204.184.146) with same results. Thanks!

  9. Thank you very much, managed to get my service installed and running between to raspberry pi’s – one in the US and one in Canada. Now, I’m looking for an easy way to restart haproxy when I make changes. Any advice is appreciated.

    Steve

  10. Jan,

    thanks a lot for this guide – I got everything setup and working without a problem. However, as of yesterday Netflix thinks I’m based in UK. Has anything changed? How exactly does Netflix determine my location?

    Cheers,
    Martin

  11. Alessandro, the server on that IP address is running the poor-mans configuration using a single IP address. And you’re right about your 2nd question.

    Cheers,
    Jan

  12. I’m in the same boat as others. I have a tomato router but, only want certain traffic at certain times, from specified PCs to go to US sites as needed. I have set up BIND9 on my VPS and now thanks to Jan :) haproxy. The only thing I would use my VPS for at the moment would be the DNS/Proxying browser wise.

    I know a lot (ok ALL) of us noobs would greatly appreciate a how-to for dummies from ye olde linux gawds :)

  13. @djvdorp: Another idea is to add the IP addresses and domain names to the /etc/hosts file (Linux/Mac), there is an equivalent for Windows as well. However, this would only work on your computer.

    Cheers,
    Jan

  14. Get yourself a router and install Tomato. Dnsmasq is included. Eg Asus RT-N66U or RT-AC66U and there are even cheaper ones.

  15. @Jan:

    Thanks for the info and pointing out that my proposed solution wouldnt be ideal (my VPS is across the ocean indeed).

    Do you happen to know other solutions when not running an in-home Pi for DNS tunneling? Can it be done on router / modem level for example? (dont have a tomato one unfortunately)

  16. Right, I understand. But what is the difference between “address” and “server” in dnsmasq on the (local) router?

  17. Some websites require DNS unblocking on the root domain level. This is my attempt to not route every subdomain to my proxy since it’s not necessary for unblocking and to conserve bandwidth.

    Cheers,
    Jan

  18. What is the difference between “address” and “server” in dnsmasq on the (local) router?

  19. Some services create random subdomains. One minute its ljgidk.subdomain.com, the other one it’s ljasdlq.subdomain.com. It is impossible to know these beforehand. That’s why I ask this ;-)

  20. HAProxy doesn’t know http://myip.nl, only http://www.myip.nl as per your HAProxy configuration. While you’re sending the entire domain myip.nl (including all sub-domains btw.) to your HAProxy server. This works in sniproxy because sniproxy wildcard-tunnels the entire domain. Tunneling an entire domain is convenient but you’re essentially creating a VPN for the domain, tunnelling all traffic from/to it through sniproxy. Not necessarily a good idea because it kills the benefits of both, Anycast and Geocast. CDN-friendly DNS unblocking will only send the essential (sub-)domains required to pass the geo-fence through your proxy server.

    Cheers,
    Jan

  21. Well I have done some testing. If I type server=/.myip.nl/MYVPS IP in dnsmasq on my router, going to http://www.myip.nl does NOT work. If I type address=/.myip.nl/MYVPS IP, it does work. (of course, I have previously added http://www.myip.nl to the haproxy.conf file). This is odd, because when I use SNIProxy, “server=” works just fine. Any thoughts, Jan?

  22. You can use my Dnsmasq setup on a remote server as well. However, I don’t think that’s a good idea since it increases latency for every DNS query a lot, particularly if there’s an ocean between you and the DNS server. Same applies to BIND btw.

    Using a remote DNS server also messes with services which rely on Geocast (http://en.wikipedia.org/wiki/Geocast). As a much cheaper alternative to Anycast, the destination IP address of a geocasted service is determined by the geo-location of the DNS server. This can wreak havoc on your download rate! I’m sorry if this is too much geek speak but I’m all about bandwidth when it comes to DNS unblocking ;-)

    Cheers,
    Jan

Comments are closed.