DNS unblocking using Dnsmasq and HAProxy

36 thoughts on “DNS unblocking using Dnsmasq and HAProxy”

1. 

   tetrahost

   July 22, 2016 at 21:57

   Thanks for this useful post
2. 

   Anthony Ettinger

   February 13, 2016 at 07:05

   Can someone point me to an article that allows me to do this using DNSMasq and SNI Proxy instead of HAProxy?
   • 

     Alessandro

     February 13, 2016 at 07:09

     I think you just need to point DNSMasq for the correct domains to the server running SNI proxy and you’re done
3. 

   Jan

   August 11, 2015 at 06:56

   I’ve written an article in German on how to do this with DNSmasq, HAproxy, Ubuntu and also with some OpenWrt flavor: SmartDNS with HAproxy and DNSmasq (in German).

   Greetings, Jan
4. 

   Jan

   January 20, 2015 at 15:28

   If “Ed” is reading this: please resend me your email address using the contact form, the address got lost somehow in the contact form sent to me.

   Cheers
   Jan
5. 

   Mike

   November 2, 2014 at 06:30

   My setup is dnsmasq and haproxy running on VPS, and just setting my DNS of the media PC to the VPS since I get 20ms ping times and I don’t use my HomeServer for general browsing.

   Has anyone scripted anything to make it quick and easy to add more domains to the haproxy and dnsmasq files? I like plink (from putty) to automate stuff with very hacky and basic windows .bat files. Stuff like appending stuff to a file is cake, but adding stuff in the middle of files (haproxy conf) is out of my league. Something would need to sed or keep track of the last rule for each section?

   Thanks very much for putting together these informative posts!
   • 

     Mike

     November 2, 2014 at 08:44

     FYI, I’m going to skip automating it with .bat and plink, I’ll just pull up the following, search and replace the last one I added, and then paste into my ssh window:

     sed -i.bkp ‘/backend b_catchall_http$/i \
     use_backend b_catchall_http if { hdr(host) -i http://www.starzplay.com }\
     ‘ /etc/haproxy/haproxy.cfg

     sed -i.bkp ‘/frontend f_catchall_https$/i \
     use-server http://www.starzplay.com if { hdr(host) -i http://www.starzplay.com }\
     server http://www.starzplay.com http://www.starzplay.com:80 check inter 10s fastinter 2s downinter 2s fall 1800\
     ‘ /etc/haproxy/haproxy.cfg

     sed -i.bkp ‘/backend b_catchall_https$/i \
     use_backend b_catchall_https if { req_ssl_sni -i http://www.starzplay.com }\
     ‘ /etc/haproxy/haproxy.cfg

     sed -i.bkp ‘/^backend b_deadend_http$/i \
     use-server http://www.starzplay.com if { req_ssl_sni -i http://www.starzplay.com }\
     server http://www.starzplay.com http://www.starzplay.com:443 check inter 10s fastinter 2s downinter 2s fall 1800\
     ‘ /etc/haproxy/haproxy.cfg
     /etc/init.d/haproxy restart

     Note: I couldn’t figure out how to prevent some extra blank lines, but I’ve decided I could live with it and eventually manually clean it up.
     • 

       Mike

       November 2, 2014 at 08:50

       Oh geez, I feel stupid.

       So for anyone reading this, the correct approach would have been to add it to the config.json and regenerate the files, copy them to the right places and restart services.

       D’oh. It’s done for me anyways.
6. 

   yousuf

   September 16, 2014 at 09:58

   got everything working right, have the non-sni based setup
   all the sites work as they should, including xfinityTV, however the liveTV section of xfinitytv isnt working, im guessing they will need to be added to config.json

   if anyone can give me the the few lines to add i would appreciate the help
7. 

   Juan

   August 31, 2014 at 03:55

   Does any body has the configuration for hbogo and showtime anywhere?
8. 

   Peter

   July 17, 2014 at 13:02

   This is great. I now have a cheaper solution for Netflix (which is 99% of my use).
   I also like to be able to occasionally rent some new release movies when they are not yet available in Australia. Vudu doesn’t seem to work with this setup. I did wireshark the transaction using unblock-us to see what goes through and what doesn’t. I seemed to be able to get to the watching point and got a black screen. Is there another service that I can use that allows this (preferrably available on AppleTV or PS3) that I can use this setup for?
9. 

   johnny

   July 5, 2014 at 07:54

   Jan- thank you for writing this up — but it falls short of my goal, which is your anti-goal. I want to route *all* netflix traffic through this and not just the geofenced features.

   Why? Verizon/Comcast are intentionally letting Netflix underperform and routing traffic through a VPN/remote server overcomes the problem. So instead of logging into a VPN, I’d love to route my traffic through a remote VPS.

   Could you help me identify the correct configuration to route all netflix.com traffic to the VPS?
10. 

    Sherif

    May 30, 2014 at 15:28

    Hello,
    Thanks for this invention :)
    I have the same question as fred, how to do the root level unblocking, I believe haproxy wont be able to do that, right?
    BR, Sherif.
11. 

    apuredol

    April 19, 2014 at 05:04

    Is it true that sni pure works with ios devices? I am trying to run pandora and it works in the browser, but in my iphone it says “Cannot connect to Pandora”; I tried my vps server configuration and yours (199.204.184.146) with same results. Thanks!
12. 

    Steve

    April 15, 2014 at 05:40

    wonderful, thank you.

    Steve
13. 

    Jan

    April 12, 2014 at 00:01

    Here’s how I do it:
    
curl https://raw2.github.com/exceliance/haproxy/master/scripts/haproxy.init.debian > /etc/init.d/haproxy
chmod +x /etc/init.d/haproxy
update-rc.d haproxy defaults


    You may have to edit the start script and change the paths.

    Cheers,
    Jan
14. 

    Steve

    April 11, 2014 at 20:31

    Thank you very much, managed to get my service installed and running between to raspberry pi’s – one in the US and one in Canada. Now, I’m looking for an easy way to restart haproxy when I make changes. Any advice is appreciated.

    Steve
15. 

    Martin

    March 28, 2014 at 12:08

    Jan,

    thanks a lot for this guide – I got everything setup and working without a problem. However, as of yesterday Netflix thinks I’m based in UK. Has anything changed? How exactly does Netflix determine my location?

    Cheers,
    Martin
16. 

    Jan

    March 26, 2014 at 20:12

    Rodrigo, I used Fiddler and tcpdump.

    Cheers,
    Jan
17. 

    Rodrigo

    March 26, 2014 at 19:36

    Jan,

    Congrats, nice work.

    How do u find out what domains and subdomains you should add???

    For the moment I have found this tool https://github.com/guelfoweb/knock , but when using it, the list is pretty large…

    Thanks
    Rodrigo
18. 

    Jan

    March 22, 2014 at 17:14

    Alessandro, the server on that IP address is running the poor-mans configuration using a single IP address. And you’re right about your 2nd question.

    Cheers,
    Jan
19. 

    Alessandro Tagliapietra

    March 22, 2014 at 17:01

    Hi, thanks for the nice post, what’s the 199.204.184.146 ip is for? The haproxy server running https://github.com/trick77/tunlr-style-dns-unblocking/blob/master/haproxy.conf ?

    But that’s the single ip configuration right? For multiple ip I would need to change each domain ip to the corresponding haproxy frontend configuration right?
20. 

    Dave

    March 7, 2014 at 21:32

    I’m in the same boat as others. I have a tomato router but, only want certain traffic at certain times, from specified PCs to go to US sites as needed. I have set up BIND9 on my VPS and now thanks to Jan :) haproxy. The only thing I would use my VPS for at the moment would be the DNS/Proxying browser wise.

    I know a lot (ok ALL) of us noobs would greatly appreciate a how-to for dummies from ye olde linux gawds :)
21. 

    Jan

    March 4, 2014 at 15:57

    @Fred: Address ties a domain name directly to an IP address, while server does a name server lookup.
22. 

    Jan

    March 4, 2014 at 15:52

    @djvdorp: Another idea is to add the IP addresses and domain names to the /etc/hosts file (Linux/Mac), there is an equivalent for Windows as well. However, this would only work on your computer.

    Cheers,
    Jan
23. 

    Fred

    March 4, 2014 at 15:51

    Get yourself a router and install Tomato. Dnsmasq is included. Eg Asus RT-N66U or RT-AC66U and there are even cheaper ones.
24. 

    djvdorp

    March 4, 2014 at 15:44

    @Jan:

    Thanks for the info and pointing out that my proposed solution wouldnt be ideal (my VPS is across the ocean indeed).

    Do you happen to know other solutions when not running an in-home Pi for DNS tunneling? Can it be done on router / modem level for example? (dont have a tomato one unfortunately)
25. 

    Fred

    March 4, 2014 at 14:00

    Right, I understand. But what is the difference between “address” and “server” in dnsmasq on the (local) router?
26. 

    Jan

    March 4, 2014 at 13:40

    Some websites require DNS unblocking on the root domain level. This is my attempt to not route every subdomain to my proxy since it’s not necessary for unblocking and to conserve bandwidth.

    Cheers,
    Jan
27. 

    Fred

    March 4, 2014 at 13:07

    What is the difference between “address” and “server” in dnsmasq on the (local) router?
28. 

    Fred

    March 4, 2014 at 12:37

    Some services create random subdomains. One minute its ljgidk.subdomain.com, the other one it’s ljasdlq.subdomain.com. It is impossible to know these beforehand. That’s why I ask this ;-)
29. 

    Jan

    March 4, 2014 at 12:16

    HAProxy doesn’t know http://myip.nl, only http://www.myip.nl as per your HAProxy configuration. While you’re sending the entire domain myip.nl (including all sub-domains btw.) to your HAProxy server. This works in sniproxy because sniproxy wildcard-tunnels the entire domain. Tunneling an entire domain is convenient but you’re essentially creating a VPN for the domain, tunnelling all traffic from/to it through sniproxy. Not necessarily a good idea because it kills the benefits of both, Anycast and Geocast. CDN-friendly DNS unblocking will only send the essential (sub-)domains required to pass the geo-fence through your proxy server.

    Cheers,
    Jan
30. 

    Fred

    March 4, 2014 at 11:51

    Well I have done some testing. If I type server=/.myip.nl/MYVPS IP in dnsmasq on my router, going to http://www.myip.nl does NOT work. If I type address=/.myip.nl/MYVPS IP, it does work. (of course, I have previously added http://www.myip.nl to the haproxy.conf file). This is odd, because when I use SNIProxy, “server=” works just fine. Any thoughts, Jan?
31. 

    Fred

    March 4, 2014 at 11:12

    What’s the difference between address and server in dnsmasq?
32. 

    Jan

    March 3, 2014 at 19:20

    You can use my Dnsmasq setup on a remote server as well. However, I don’t think that’s a good idea since it increases latency for every DNS query a lot, particularly if there’s an ocean between you and the DNS server. Same applies to BIND btw.

    Using a remote DNS server also messes with services which rely on Geocast (http://en.wikipedia.org/wiki/Geocast). As a much cheaper alternative to Anycast, the destination IP address of a geocasted service is determined by the geo-location of the DNS server. This can wreak havoc on your download rate! I’m sorry if this is too much geek speak but I’m all about bandwidth when it comes to DNS unblocking ;-)

    Cheers,
    Jan
33. 

    djvdorp

    March 3, 2014 at 13:54

    Thanks for this useful post as well, was kinda waiting for it! Any chance you can also help people out who are willing to use BIND instead of DNSmasq (eg I dont run my DNS server in my house, but would like to use the HAProxy box as DNS server as well) as they do here:
    http://corporate-gadfly.github.....dns-server