strongSwan 5 based IPSec VPN, Ubuntu 14.04 LTS and PSK/XAUTH

I prefer strongSwan over Openswan because it’s still in active development, easier to setup and doesn’t require a L2TP daemon. I prefer a simple IKEv1 setup using PSK and XAUTH over certificates. If you plan to share your VPN server with your friends it’s also a lot easier to setup for them without certificates. I haven’t tried the VPN configuration below with non-Apple clients but it works well with iOS and OS X clients. Make sure to use the Cisco IPSec VPN profile, not the L2TP over IPSec profile you need for Openswan. While strongSwan works well with KVM and Xen containers, it probably won’t work with non-virtualised containers like OpenVZ or LXC. Continue reading

LXC 1.0 Web Panel for Ubuntu 14.04

LXC is awesome! You can create and start your own virtual container with just 3 commands in Ubuntu 14.04.

apt-get install lxc debootstrap lxc-templates
lxc-create -t ubuntu -n demo
lxc-start -n demo -d

It doesn’t get any easier than this. There’s even a Boostrap-based fronted available: LXC Web Panel.

lxc-web-panel

Unfortunately, LXC Web Panel doesn’t work with LXC 1.0 which is part of Ubuntu 14.04. Fortunately though, there’s a fork available on GitHub which adds support for LXC 1.0:

https://github.com/claudyus/LXC-Web-Panel

I re-forked claudyus’ LXC Web Panel fork and added support for Ubuntu 14.04 and a few other things. My forked fork is available here: https://github.com/trick77/LXC-Web-Panel Claudyus has already updated his repository with my changes.

By the way, the original author of LXC Web Panel said he’s currently working on a Bootstrap 3 based version for LXC 1.0 which will include a RESTful API and other new features. Make sure to follow this guy on GitHub: https://github.com/googley

DNS unblocking setup tester

This may help setting up your own DNS unblocking solution:

http://trick77.com/dns-unblocking-setup-tester/

Once everything has been set up properly, all ticks should be green like in this screenshot:

tester

I just pushed another update to GitHub, please make sure to use a configuration generated with the latest generator version or the tester will fail. My main motivation to create this tester was to reduce the amount of support requests I’m receiving. Let’s see how well this goes :-)

Tomcat freezes while starting in Ubuntu 14.04 LTS

After upgrading one of my KVMs to Ubuntu Server 14.04 LTS, Tomcat 7 started to freeze while starting up with:
INFO: Deploying configuration descriptor /etc/tomcat7/Catalina/localhost/ROOT.xml

Only after several minutes, Tomcat generates the following message and starts accepting requests:
INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [295,490] milliseconds.
INFO: Server startup in 296882 ms

If you don’t have a requirement for strong cryptography in Tomcat, you might as well switch to the less secure non-blocking /dev/urandom source instead of /dev/random.
Create a file named setenv.sh in /var/lib/tomcat7/bin make it executable:
#!/bin/sh
export CATALINA_OPTS="-Djava.security.egd=file:/dev/./urandom"

..and restart Tomcat.

It will now restart within seconds thanks to the non-blocking random source.

Supermicro NTP DDoS Vulnerability

I received a notification that one of my dedicated servers was taking part in a NTP based DDoS reflection attack. At first I was like “No way!” since I don’t use NTP on any servers. Closer inspection of the source IP address revealed that the attack was coming from my Supermicro server’s built in IPMI controller. And indeed, Supermicro is using a vulnerable NTP version on its IPMI controllers:

ntpdc -n -c monlist ipmi.mysupermicroserver.com
remote address port local address count m ver rstr avgint lstint
===============================================================================
186.2.161.nnn 53842 76.20.120.nn 51127 7 2 0 0 0
217.147.208.n 123 76.20.120.nn 1 4 4 0 0 7
130.60.204.nn 123 76.20.120.nn 1 4 4 0 0 8

The quickest fix is to turn NTP sync off in IPMI as described here. If for some reason you have a requirement for NTP, here’s how to fix the Supermicro firmware on your own (not for the faint-hearted!).

Since Supermicro has a spotty track record when it comes to IPMI controller security, it’s highly recommended to define a set of jump hosts in the IP Access Control menu. Here’s a gotcha: the default policy is set to ACCEPT which means you have to add a DROP rule at the end with 0.0.0.0/0. Obviously, a private VLAN would be the preferred way, but if no VLAN is available, IP access control comes in handy. The IP access control list will filter any traffic to the IPMI controller except for the defined IP ranges. It will block access to NTP as well.

Still waiting for Supermicro to finally fix the issue in a new firmware revision though…

DNS-unblocking configuration for CBS’s iOS app

I love watching the Late Late Show with Craig Ferguson. Here’s what’s needed to watch CBS content on iPad outside the U.S. using my DNS-unblocking config generator.

latelateshow

   {
      "name": "cbs-akamai-ipad",
      "dest_addr": "ipad-streaming.cbs.com",
      "modes": [
        {
          "port": 80,
          "mode": "http"
        },
        {
          "port": 443,
          "mode": "https"
        }
      ],
      "catchall": true,
      "enabled": true
    }

Add this to config.json, regenerate the configuration files and make sure to upload them to the right places. As with NBC’s iOS app, the video stream itself is geo-fenced which may lead to considerable bandwidth consumption on your VPS server.

By the way, is it just me or does audio quality suck badly in CBS’s iOS app?

DNS-unblocking configuration for NBC’s iOS app

Today, NBC updated its mobile app for iPad and iPhone. The latest version features AirPlay support which means you can watch NBC TV shows on a large TV if you have an Apple TV connected to it. NBC offers free (ad-supported) content in its iOS app, including:

  • The Tonight Show Starring Jimmy Fallon
  • Late Night With Seth Myers
  • About A Boy
  • Grimm
  • The Blacklist
  • The Voice
  • …and many more

photo

In order to use the NBC app on iPad or iPhone outside the U.S., a new configuration entry is required in my non-SNI DNS-unblocking config generator’s config.json file:

    {
      "name": "nbc-ios",
      "dest_addr": "tve_nbc-vh.akamaihd.net",
      "modes": [
        {
          "port": 80,
          "mode": "http"
        },
        {
          "port": 443,
          "mode": "https"
        }
      ],
      "catchall": true,
      "enabled": true
    }

Since NBC uses a geo-fenced video stream from Akamai, the entire content stream has to be proxied through HAProxy which may – depending on your usage – lead to considerable bandwidth usage on your remote VPS server.

Fulltext search for Tiny Tiny RSS (TTRSS) with Sphinx and MySQL in Debian/Ubuntu

I haven’t found a working tutorial on setting up Sphinx fulltext search for the awesome Tiny Tiny RSS Reader (TTRSS) and MySQL in Debian/Ubuntu. So, without further ado, here it is:

apt-get install sphinxsearch

Create /etc/sphinxsearch/sphinx.conf:

source ttrss
{
	type			= mysql
	sql_host		= localhost
	sql_user		= ttrss
	sql_pass		= changeme
	sql_db			= ttrss
	sql_port		= 3306
	sql_query_pre		= SET NAMES utf8

	# UNIX socket name
	# optional, default is empty (reuse client library defaults)
	# usually '/var/lib/mysql/mysql.sock' on Linux
	# usually '/tmp/mysql.sock' on FreeBSD
	#
	# sql_sock		= /var/lib/mysql/mysql.sock

        sql_query		= \
		SELECT int_id AS id, ref_id, UNIX_TIMESTAMP() AS updated, \
 			ttrss_entries.title AS title, link, content, \
                        ttrss_feeds.title AS feed_title, \
                        marked, published, unread, \
                        author, ttrss_user_entries.owner_uid \
                        FROM ttrss_entries, ttrss_user_entries, ttrss_feeds \
                        WHERE ref_id = ttrss_entries.id AND feed_id = ttrss_feeds.id;


	sql_attr_uint		= owner_uid
	sql_attr_uint		= ref_id

	sql_ranged_throttle	= 0

	sql_query_info		= \
		SELECT * FROM ttrss_entries,  \
			ttrss_user_entries WHERE ref_id = id AND int_id=$id


}

source delta : ttrss 
{
        sql_query		= \
                SELECT int_id AS id, ref_id, UNIX_TIMESTAMP() AS updated, \
                        ttrss_entries.title AS title, link, content, \
                        ttrss_feeds.title AS feed_title, \
                        marked, published, unread, \
                        author, ttrss_user_entries.owner_uid \
                        FROM ttrss_entries, ttrss_user_entries, ttrss_feeds \
                        WHERE ref_id = ttrss_entries.id AND feed_id = ttrss_feeds.id \
                        AND ttrss_entries.updated > UNIX_TIMESTAMP() - INTERVAL 24 HOUR;

        sql_query_killlist      = \
		SELECT int_id FROM ttrss_entries, ttrss_user_entries \
                	WHERE ref_id = ttrss_entries.id AND updated > UNIX_TIMESTAMP() - INTERVAL 24 HOUR;

}

index ttrss
{
        source			= ttrss
	path			= /var/lib/sphinxsearch/data/ttrss
	docinfo			= extern
	mlock			= 0
	morphology		= none
	min_word_len		= 1
	charset_type		= utf-8
	min_prefix_len	        = 3
	prefix_fields		= title, content, feed_title, author
	enable_star		= 1
	html_strip		= 1

}

index delta : ttrss 
{
	source			= delta
	path			= /var/lib/sphinxsearch/data/ttrss_delta
}

indexer
{
	mem_limit		= 32M
}

searchd
{
	listen			= 127.0.0.1:9312

	log			= /var/log/sphinxsearch/searchd.log
	query_log		= /var/log/sphinxsearch/query.log
	read_timeout		= 5
	client_timeout		= 300
	max_children		= 30
	pid_file		= /var/run/sphinxsearch/searchd.pid
	max_matches		= 1000
	seamless_rotate		= 1
	preopen_indexes		= 1
	unlink_old		= 1
	mva_updates_pool	= 1M
	max_packet_size		= 8M
	max_filters		= 256
	max_filter_values	= 4096
	compat_sphinxql_magics  = 0
}

Create the indices using indexer --all
Set START=yes in /etc/default/sphinxsearch and start Sphinx using service sphinxsearch start

The last step is to enable Sphinx search in TTRSS:

	// *********************
	// *** Sphinx search ***
	// *********************

	define('SPHINX_ENABLED', true);
	// Enable fulltext search using Sphinx (http://www.sphinxsearch.com)
	// Please see http://tt-rss.org/wiki/SphinxSearch for more information

And that is it. Lighting-fast full-text search in TTRSS! Probably only useful if you have a lot of feeds/articles and you’re keeping them for quite while before purging them to oblivion.

Digitec Tagesaktionen (Daily Deals) RSS Feed

Completely irrelevant to y’all unless you’re living in Switzerland and you’re into tech gadgets:

http://feeds.wolke7.me/digitec/feed.xml

The feed will show the current “Tagesaktion” (daily deals) from digitec.ch as a RSS 2.0 feed so you don’t have to think about visiting digitec’s site every day. I missed a pretty decent SSD deal the other day but from now on, the daily deals are showing up in my RSS reader Tiny Tiny RSS – which is awesome btw.

ttrss

Apache2 2.4+ not logging remote IP address using mod_remoteip

Since there’s no support for mod_rpaf in Apache2 2.4+ it’s recommended to use mod_remoteip instead if Apache2 is running behind a proxy like HAProxy.

mod_remoteip can be enabled using

a2enmod remoteip

It can be fine tuned in ./conf-available/remoteip.conf. You have to manually create the file if it doesn’t exist.

RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy 127.0.0.1

And don’t forget to

cd ./conf-enabled
ln -s ../conf-available/remoteip.conf
service apache2 restart

While the remote IP address is injected into PHP just fine, Apache2 continues to log 127.0.0.1 as the remote IP address in access.log.

A modification is required in apache2.conf:

#LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

%h has to be replaced with %a. I’m not sure if this is a bug or not but Apache2 will log the real remote IP address from now on.

If you’re still seeing 127.0.0.1 then maybe your proxy doesn’t forward the remote IP address to Apache2. For HAProxy, use the forwardfor option:

  option        forwardfor

Netflix DNS-unblocking without SNI for your Xbox 360, PS3, WDTV, Samsung TV

My poor man’s DNS-unblocking configuration using just a single, public IP address has one serious limitation: it will not run Netflix or Hulu Plus with non-SNI players like the PS3, Xbox 360, Samsung TVs, Sony BluRay players and possibly quite a few other devices. A commenter (kudos go out to Alex) suggested to use Netfilter’s DNAT port forwarding mechanism to overcome this limitation. Using DNAT you can forward packets based on the source-ip:port to a remote-ip:port.

So, here’s a modified version of the poor man’s DNS-unblocking approach. You will need some sort of Linux server at home to do this. I’m using a Raspberry Pi Linux mini computer which is up 24/7 on my LAN. And of course you will need a remote Linux server with an IP address registered in the U.S. You can get a low-end virtual private server for as low as $5/year. Unfortunately, it’s almost impossible to come up with a step-by-step tutorial because every LAN setup is different, hence you have to have some Linux and networking skills in order to get this baby up and running.

And here’s how this approach works: A DNS forwarder like Dnsmasq on your local Linux server will intercept domain names relevant for DNS unblocking. All other queries will be forwarded to the DNS resolver/forwarder of your choice (usually, this will be your router). The intercepted domain names will be resolved to IP addresses which are routed to your Linux server within your LAN. Depending on the resolved IP addresses and ports, iptables DNAT rules will forward the request to a HAProxy proxy on your remote server. Each domain name can have its own internal IP adress and thus its own listening port on your remote server’s HAProxy. And since every domain name can have it’s own HAProxy TCP proxy on your remote server, there’s no need for SNI! Continue reading

DNS unblocking using Dnsmasq and HAProxy

As I mentioned in my previous post, the open source DNS forwarder Dnsmasq is ideal for the DNS part of DNS unblocking. I’m running Dnsmasq on a $30 Raspberry Pi credit card sized mini computer which is up 24/7 anyway since it also handles all VOIP phone calls at home. I point my Mac, Apple TV and iPad to the RPi as the primary DNS server.
On the server side, I’ve setup a HAProxy instance using just a single IP address as a proof of concept. This poor-man’s approach works beautifully with SNI-capable devices like my Mac and iOS devices. I think newer Android devices are SNI-compatible as well but I haven’t tested it. Windows 7 and up should be OK too. Older devices like the Playstation 3 or Xbox 360 are most likely not SNI-compatible and won’t work with my highly cost-efficient single IP address approach. Unfortunately, even some of the newest multimedia players don’t support SNI.

The HAProxy server is running on a lowend virtual private server in the U.S. As a starting point, feel free to use my proof of concept server as shown in the Dnsmasq configuration below. In the web browser, you should be able to watch Netflix, Hulu/HuluPlus, free episodes/TV shows on MTV, Disney XD, Syfy, NBC, ABC, Vevo, Crackle, PBS and CWTV. Netflix works on iPad and Apple TV too. HuluPlus could work on iOS as well. Continue reading

Tunlr-style DNS unblocking for Pandora, Netflix, Hulu et al

Since Tunlr closed down unexpectedly this week, I decided to publish my ideas and findings on the subject of DNS unblocking. I used Tunlr for some time when I decided to develop my own, private DNS unblocking solution last year.

Why VPNs are no good for streaming

DNS unblocking refers to a technique used to circumvent geo-fenced Internet services without the use of a VPN. When we’re using a VPN to access geo-fenced websites, usually all our Internet traffic gets routed through a remote VPN server. With DNS unblocking, only selected traffic gets routed through a remote proxy server, ideally just the minimum traffic required to trick geo-fenced services like Pandora, Netflix or Hulu into “thinking” our current geolocation is within the United States (or any other country required to pass the geo-fence). One advantage is that DNS unblocking works for all devices that allow custom DNS settings while a VPN only works on a computer or in the router. But the big advantage over a VPN is that DNS unblocking allows the full and intended use of Content Delivery Networks (CDN).

Continue reading

Using ipset to ban bad IP addresses from Project Honey Pot, Spamhaus, Tor, OpenBL and more

ipset in combination with iptables are the perfect tools to ban thousands of blacklisted IP addresses from IP blacklist providers like Project Honey Pot, Spamhaus, OpenBL and virtually anyone providing a list of “bad” IP addresses from a Linux server.

I’ve created a very simple Bash shell script which can be used to auto-update blacklisted IP addresses. Please see the documentation in the README.md on how to install it. It doesn’t matter if the blacklist comes as a raw IP list, as XML or CSV. The script will find any IPv4 including IPv4 with network prefixes (CIDR notation).

Currently, the script downloads blacklisted IP addresses from the following blacklists:

  1. Project Honey Pot
  2. Tor Exit Nodes
  3. MaxMind Anonymous Proxies
  4. BruteForceBlocker IP list from danger.rulez.sk
  5. Emerging Threats list from emergingthreats.net
  6. Spamhaus Don’t Route Or Peer List
  7. C.I. Army Malicious IP list
  8. OpenBL 30 day list
  9. Autoshun’s Shun list

Link to the git repository: github.com/trick77/ipset-blacklist

So, is there any benefit in banning those IP addresses? Well, it certainly reduces comment spam on a WordPress blog and there have been claims from websites owners that their servers had been attacked through Tor. The number of comment spam attempts on this blog dropped quite impressively after implementing the IP address bans:

antispam1