DNS unblocking using Dnsmasq and HAProxy

1. Thanks for this useful post

2. Can someone point me to an article that allows me to do this using DNSMasq and SNI Proxy instead of HAProxy?

   1. I think you just need to point DNSMasq for the correct domains to the server running SNI proxy and you’re done

3. I’ve written an article in German on how to do this with DNSmasq, HAproxy, Ubuntu and also with some OpenWrt flavor: SmartDNS with HAproxy and DNSmasq (in German).

   Greetings, Jan

4. If “Ed” is reading this: please resend me your email address using the contact form, the address got lost somehow in the contact form sent to me.

   Cheers
   Jan

5. My setup is dnsmasq and haproxy running on VPS, and just setting my DNS of the media PC to the VPS since I get 20ms ping times and I don’t use my HomeServer for general browsing.

   Has anyone scripted anything to make it quick and easy to add more domains to the haproxy and dnsmasq files? I like plink (from putty) to automate stuff with very hacky and basic windows .bat files. Stuff like appending stuff to a file is cake, but adding stuff in the middle of files (haproxy conf) is out of my league. Something would need to sed or keep track of the last rule for each section?

   Thanks very much for putting together these informative posts!

   1. FYI, I’m going to skip automating it with .bat and plink, I’ll just pull up the following, search and replace the last one I added, and then paste into my ssh window:

      sed -i.bkp ‘/backend b_catchall_http$/i \
      use_backend b_catchall_http if { hdr(host) -i http://www.starzplay.com }\
      ‘ /etc/haproxy/haproxy.cfg

      sed -i.bkp ‘/frontend f_catchall_https$/i \
      use-server http://www.starzplay.com if { hdr(host) -i http://www.starzplay.com }\
      server http://www.starzplay.com http://www.starzplay.com:80 check inter 10s fastinter 2s downinter 2s fall 1800\
      ‘ /etc/haproxy/haproxy.cfg

      sed -i.bkp ‘/backend b_catchall_https$/i \
      use_backend b_catchall_https if { req_ssl_sni -i http://www.starzplay.com }\
      ‘ /etc/haproxy/haproxy.cfg

      sed -i.bkp ‘/^backend b_deadend_http$/i \
      use-server http://www.starzplay.com if { req_ssl_sni -i http://www.starzplay.com }\
      server http://www.starzplay.com http://www.starzplay.com:443 check inter 10s fastinter 2s downinter 2s fall 1800\
      ‘ /etc/haproxy/haproxy.cfg
      /etc/init.d/haproxy restart

      Note: I couldn’t figure out how to prevent some extra blank lines, but I’ve decided I could live with it and eventually manually clean it up.

      1. Oh geez, I feel stupid.

         So for anyone reading this, the correct approach would have been to add it to the config.json and regenerate the files, copy them to the right places and restart services.

         D’oh. It’s done for me anyways.

6. got everything working right, have the non-sni based setup
   all the sites work as they should, including xfinityTV, however the liveTV section of xfinitytv isnt working, im guessing they will need to be added to config.json

   if anyone can give me the the few lines to add i would appreciate the help

7. Does any body has the configuration for hbogo and showtime anywhere?

8. This is great. I now have a cheaper solution for Netflix (which is 99% of my use).
   I also like to be able to occasionally rent some new release movies when they are not yet available in Australia. Vudu doesn’t seem to work with this setup. I did wireshark the transaction using unblock-us to see what goes through and what doesn’t. I seemed to be able to get to the watching point and got a black screen. Is there another service that I can use that allows this (preferrably available on AppleTV or PS3) that I can use this setup for?

9. Jan- thank you for writing this up — but it falls short of my goal, which is your anti-goal. I want to route *all* netflix traffic through this and not just the geofenced features.

   Why? Verizon/Comcast are intentionally letting Netflix underperform and routing traffic through a VPN/remote server overcomes the problem. So instead of logging into a VPN, I’d love to route my traffic through a remote VPS.

   Could you help me identify the correct configuration to route all netflix.com traffic to the VPS?

10. Hello,
    Thanks for this invention :)
    I have the same question as fred, how to do the root level unblocking, I believe haproxy wont be able to do that, right?
    BR, Sherif.

11. Is it true that sni pure works with ios devices? I am trying to run pandora and it works in the browser, but in my iphone it says “Cannot connect to Pandora”; I tried my vps server configuration and yours (199.204.184.146) with same results. Thanks!

12. wonderful, thank you.

    Steve

13. Here’s how I do it:
    
curl https://raw2.github.com/exceliance/haproxy/master/scripts/haproxy.init.debian > /etc/init.d/haproxy
chmod +x /etc/init.d/haproxy
update-rc.d haproxy defaults


    You may have to edit the start script and change the paths.

    Cheers,
    Jan

14. Thank you very much, managed to get my service installed and running between to raspberry pi’s – one in the US and one in Canada. Now, I’m looking for an easy way to restart haproxy when I make changes. Any advice is appreciated.

    Steve

15. Jan,

    thanks a lot for this guide – I got everything setup and working without a problem. However, as of yesterday Netflix thinks I’m based in UK. Has anything changed? How exactly does Netflix determine my location?

    Cheers,
    Martin

16. Rodrigo, I used Fiddler and tcpdump.

    Cheers,
    Jan

17. Jan,

    Congrats, nice work.

    How do u find out what domains and subdomains you should add???

    For the moment I have found this tool https://github.com/guelfoweb/knock , but when using it, the list is pretty large…

    Thanks
    Rodrigo

18. Alessandro, the server on that IP address is running the poor-mans configuration using a single IP address. And you’re right about your 2nd question.

    Cheers,
    Jan

19. Hi, thanks for the nice post, what’s the 199.204.184.146 ip is for? The haproxy server running https://github.com/trick77/tunlr-style-dns-unblocking/blob/master/haproxy.conf ?

    But that’s the single ip configuration right? For multiple ip I would need to change each domain ip to the corresponding haproxy frontend configuration right?

20. I’m in the same boat as others. I have a tomato router but, only want certain traffic at certain times, from specified PCs to go to US sites as needed. I have set up BIND9 on my VPS and now thanks to Jan :) haproxy. The only thing I would use my VPS for at the moment would be the DNS/Proxying browser wise.

    I know a lot (ok ALL) of us noobs would greatly appreciate a how-to for dummies from ye olde linux gawds :)

21. @Fred: Address ties a domain name directly to an IP address, while server does a name server lookup.

22. @djvdorp: Another idea is to add the IP addresses and domain names to the /etc/hosts file (Linux/Mac), there is an equivalent for Windows as well. However, this would only work on your computer.

    Cheers,
    Jan

23. Get yourself a router and install Tomato. Dnsmasq is included. Eg Asus RT-N66U or RT-AC66U and there are even cheaper ones.

24. @Jan:

    Thanks for the info and pointing out that my proposed solution wouldnt be ideal (my VPS is across the ocean indeed).

    Do you happen to know other solutions when not running an in-home Pi for DNS tunneling? Can it be done on router / modem level for example? (dont have a tomato one unfortunately)

25. Right, I understand. But what is the difference between “address” and “server” in dnsmasq on the (local) router?

26. Some websites require DNS unblocking on the root domain level. This is my attempt to not route every subdomain to my proxy since it’s not necessary for unblocking and to conserve bandwidth.

    Cheers,
    Jan

27. What is the difference between “address” and “server” in dnsmasq on the (local) router?

28. Some services create random subdomains. One minute its ljgidk.subdomain.com, the other one it’s ljasdlq.subdomain.com. It is impossible to know these beforehand. That’s why I ask this ;-)

29. HAProxy doesn’t know http://myip.nl, only http://www.myip.nl as per your HAProxy configuration. While you’re sending the entire domain myip.nl (including all sub-domains btw.) to your HAProxy server. This works in sniproxy because sniproxy wildcard-tunnels the entire domain. Tunneling an entire domain is convenient but you’re essentially creating a VPN for the domain, tunnelling all traffic from/to it through sniproxy. Not necessarily a good idea because it kills the benefits of both, Anycast and Geocast. CDN-friendly DNS unblocking will only send the essential (sub-)domains required to pass the geo-fence through your proxy server.

    Cheers,
    Jan

30. Well I have done some testing. If I type server=/.myip.nl/MYVPS IP in dnsmasq on my router, going to http://www.myip.nl does NOT work. If I type address=/.myip.nl/MYVPS IP, it does work. (of course, I have previously added http://www.myip.nl to the haproxy.conf file). This is odd, because when I use SNIProxy, “server=” works just fine. Any thoughts, Jan?

31. What’s the difference between address and server in dnsmasq?

32. You can use my Dnsmasq setup on a remote server as well. However, I don’t think that’s a good idea since it increases latency for every DNS query a lot, particularly if there’s an ocean between you and the DNS server. Same applies to BIND btw.

    Using a remote DNS server also messes with services which rely on Geocast (http://en.wikipedia.org/wiki/Geocast). As a much cheaper alternative to Anycast, the destination IP address of a geocasted service is determined by the geo-location of the DNS server. This can wreak havoc on your download rate! I’m sorry if this is too much geek speak but I’m all about bandwidth when it comes to DNS unblocking ;-)

    Cheers,
    Jan

33. Thanks for this useful post as well, was kinda waiting for it! Any chance you can also help people out who are willing to use BIND instead of DNSmasq (eg I dont run my DNS server in my house, but would like to use the HAProxy box as DNS server as well) as they do here:
    http://corporate-gadfly.github.io/Tunlr-Clone/#your-own-dns-server

Comments are closed.