Slow oh-my-zsh loading on macOS 11

Posted on by Jan

At some point, starting a zsh (I’m using iTerm2 here) took almost 10 seconds on macOS 11 Big Sur. This only happened the first time when I opened iTerm2 after a reboot. After mentally going through all the software changes I had done to my rig, I was pretty sure this happened right after installing Xcode. After some debugging in the oh-my-zsh code I found the culprit: it’s related to Apple’s git version which comes with Xcode. After installing the Homebrew version of git, the delay no longer occurred. It’s important though that the Homebrew git binary is found first when calling git from the command line.

However, a few days later, the initial delay came back again. This time, the culprit was nvm (Nodejs version manager) or more specifically, adding the nvm variables to my ~/.zshrc. I’ve found a snippet that lazy loads the nvm.sh in ~/.zshrc. Check out this Gist:

The two workarounds combined drastically reduce (oh-my-)zsh loading times on my Mac. Make sure to remove an existing nvm init variables from ~/.zshrc before applying the lazy loading code.

GA-Z97X-UD5H and macOS 11 Big Sur with OpenCore

Posted on by Jan

Interestingly, my 6 year old Hackintosh rig is still going strong and so I decided to upgrade it from macOS X Mojave to macOS 11 Big Sur. Since OpenCore is pretty well documented by now, I wanted to give it a try and replace Clover (which seems to use OpenCore as well by now).
However, the transition from Clover to OpenCore wasn’t smooth at all. OpenCore has quite a steep learning curve and I spent almost an entire, foggy November Sunday to get it all up and running.

The relevant system components in my Hackintosh are:

  • Gigabyte GA-Z97X-UD5H (BIOS rev. F10c)
  • Intel Haswell Core(TM) i7-4790K CPU @ 4.00GHz
  • Intel HD Graphics Intel 4600
  • Onboard ALC Audio
  • Custom flashed PCI Intel Gigabit CT NIC (so, not using onboard NIC)

Super helpful were:

A few additional notes:

  • BIOS Settings as per recommendation for Haswell, see OpenCore docs.
  • The Haswell DSDT provided by Dortanina resulted in a KP, use the ones provided in the link below.
  • Use the CPUFriendFriend generator link above to create the data for the CPUFriendDataProvider. However, I’m not sure if the CPUFriend.kext actually makes a difference. I can see the CPU frequency dropping below 1 GHz in Intel’s Power Gadget though (see screenshot below) and I think the CPU fan is running a little quieter. Can be omitted when in doubt.
  • Everything related to sleep/wake is working as expected.
  • I didn’t include any kext binaries, make sure to always download the latest versions.

Check out my Github repository: github.com/trick77/opencore-ga-z97x-ud5h

The best cloud desktop solution for Linux is… Windows!?

Posted on by Jan

I like using a remote desktop work/office environment for various reasons, travelling being on of them. This is also known as a cloud desktop. Thanks to the awesome Apache Guacamole remote desktop gateway software, I can access it everywhere, just by using a web browser (and an Internet connection).

While I love Linux, it sucks when it comes to running a remote desktop server using non-commercial software. Yes, I’ve tried xrdp. While it works, the graphics performance/latency sucks even though I was using the low resource environment xfce4. Obviously there is commercial remote desktop server software like RealVNC or NoMachine but I don’t want to shell out cash for my cloud desktop and in the case of NoMachine, its proprietary NX protocol isn’t supported by Guacamole.

However, I have an unused Windows Server 2019 educational license which I can use for my cloud desktop. It uses the RDP protocol which is fully supported by Guacamole. I realise that a Windows Server installation is pretty much overkill for just a cloud desktop but hey… it’s free (in my case). Windows 10 Pro contains an RDP server as well but I haven’t tried it.

The biggest challenge was to find the right parameters for virt-install to install a Windows server on my Linux KVM host. Here’s what I came up with:

#!/bin/sh 
NAME=desktop
RAM=4096
CPU=2
IMAGE=win-server-2019.iso
VIRTIO_IMAGE=virtio-win.iso
SIZE=50G
VNC_PORT=56681
VARIANT=win2k16
VG=vg0 
MAC=02:00:00:d1:78:d9
lvcreate -L $SIZE -n lv_vm_$NAME $VG 
virt-install --connect qemu:///system --arch=x86_64 -n $NAME -r $RAM --vcpus=$CPU \
--mac=$MAC \
--cdrom /var/lib/libvirt/images/$IMAGE \
--disk path=/dev/$VG/lv_vm_$NAME,bus=virtio \
--disk path=/var/lib/libvirt/images/$VIRTIO_IMAGE,device=cdrom \
--graphics vnc,listen=127.0.0.1,port=$VNC_PORT \
--noautoconsole \
--os-type windows \
--os-variant=$VARIANT \
--network=bridge:br0,model=virtio \
--accelerate \
--noapic

In my case, the network bridge br0 from the Linux KVM host is exposed to the guest KVM. I’m using LVM for storage.

Once the KVM is up, I’m using a VNC client to complete the Windows installation. Since the VNC port isn’t exposed to the internet (deliberately), I’m using ssh port forwarding to access it to complete the installation. Something like:

ssh myhost -L56681:127.0.0.1:56681

Since Windows won’t find the required disk drivers, I’m attaching the Windows virtio driver .iso as a CD-ROM. Look for the viostor drivers during the installation process and the logical volume will finally show up in the installer. Once the installation is complete, I’m using VNC again to update the missing Ethernet drivers in the Device Manager, configure the network and that’s pretty much it.

It’s not recommended to expose the Windows remote desktop server to the Internet. Port 3389 gets brute-force attacked 24/7. I could use the same SSH port forwarding approach shown above to access my cloud desktop by forwarding port 3389 and/or firewall the RDP port so only my Guacamole server is able to access it.

How to use IPv6 on Quickline/WWZ and pfSense firewall

Posted on by Jan

Here’s how to configure your pfSense firewall for IPv6 on Quickline/WWZ. The settings may work with other ISPs too but YMMV. I’m assuming your modem is already in bridge mode and pfSense is up and running for IPv4 DHCP on the WAN interface.

Activate IPv6 and DHCP6 in the router

We’re configuring pfSense to use DHCP6 on the WAN interface to get an IPv6 prefix from the ISP.

In System → Advanced → Networking:

  • Activate Allow IPv6

In Interfaces → WAN → General Configuration:

  • IPv6 Configuration Type: DHCP6

In Interfaces → WAN → DHCP6 Client Configuration:

  • Activate Request only an IPv6 prefix
  • DHCPv6 Prefix Delegation size (according to Quickline, ask your ISP when in doubt):
    • 56 for cable modems (HF + FTTH)
    • 64 for FTTH
  • Optional but helps if something doesn’t work: Start DHCP6 client in debug mode
  • Activate Do not wait for RA
  • Optional: Activate Do not allow PD/Address release
    • May help keeping your assigned IPv6 prefix if you prefer it to be static

In Interfaces → LAN → General Configuration:

  • IPv6 Configuration Type: Track Interface

In Interfaces → LAN → Track IPv6 Interface:

  • IPv6 Interface: WAN

In Services → DHCPv6 Server & RA → Router Advertisments:

  • Router mode: Unmanaged
  • Router priority: High

You could opt to activate pfSense’s DHCPv6 server on the LAN interface and hand out a range of available IPv6 addresses from your prefix but I have no need for a DHCPv6 server on the LAN interface. Instead, I’m making the IPv6 prefix available to the LAN clients to autoconfigure themselves for IPv6. Watch out for blocked DHCPv6 connections if you enable pfSense’s DHCPv6 server and assisted/managed RA in combination with Bogon filtering.

Very important final step: reboot pfSense. I was getting error messages like transmit failed: Can’t assign requested address which where gone after a reboot.

Is it working?

Go to Status → Gateways. If pfSense was able to get an IPv6 prefix from your ISP, the WAN_DHCP6 gateway (or whatever the name you chose for the WAN interface) shold show status Online. If it’s always in state Pending then something went wrong (see Debugging below).

Use a web browser in a LAN client (check if it was assigned an IPv6, reboot when in doubt) to check if IPv6 is available and go to https://ipv6test.google.com.

Optimization

While IPv6 has been around for quite a while, most ISP and network providers still optimize routing for IPv4 (=have more IPv4 peers than IPv6 BGP peers). That’s why you might get better/faster connections when giving IPv4 precedence over IPv6 (the default is to always prefer IPv6).

That’s why I’m instructing pfSense to prefer IPv4 over IPv6 if both are available in a DNS response in System → Advanced → Networking → IPv6 Options: Activate Prefer IPv4 over IPv6.

Obviously, this setting needs to be configured in every client on your LAN (if the device supports it) since it’s based on how a DNS response is interpreted. For Linux based clients have a look at /etc/gai.conf

Debugging

If debug logging is enabled for the DHCP6 client you might find helpful debugging information in Status → System Logs → DHCP. You can use the Advanced Log Filter to search for dhcp6 messages in the log.

Do LAN clients get a public IPv6 but the IPv6 browser check still fails? Check the firewall rules for blocked IPv6 traffic.

Auto-restart crashed mining processes in ethOS 1.2.9

Posted on by Jan

ethOS 1.2.9 brings a few changes which break my auto-restart script for ethOS 1.2.7. Since 1.2.9 contains improved GPU crash detection, I rewrote the restart script to use the built-in detection mechanisms. For the required cron job please see my initial post which is available here.

As long as the DRY_RUN variable is set to false, the script won’t take any action, it just logs what it would do.

 

Auto-restart crashed mining processes in ethOS 1.2.7

Posted on by Jan

I have been running a crypto-currency mining rig on the Linux based ethOS distro for quite some time now. While I realize that ethOS is problematic license-wise, it’s still a great distro to get a mining rig up and running in almost no time. The Nvidia GPUs in my rig are well tuned to operate at their optimum cost/hashrate ratio. However, due to bugs in the miner and/or the GPU drivers, every few days one of the GPUs stops mining. Sometimes ethOS is able to recover the GPU and gets it back to mining but sometimes it doesn’t seem to detect the crashed/hanging mining process at all. This is why I added a cron job that runs every 15 minutes and checks if all GPUs are still mining. If not, the miners will be re started using the minestop and minestart commands provided by ethOS.

The cron job starts the Bash script below. If it detects a problem, it writes to the console and additionally to /tmp/rigcheck.log. It’s been running smoothly on my ethOS v1.2.7 mining rig. I recommend putting it in /home/ethos/rigcheck.sh and don’t forget to add execute permissions using chmod +x /home/ethos/rigcheck.sh

The cron job can be created like this:

cat << EOF | sudo tee /etc/cron.d/rigcheck
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
*/15 * * * *   root    /home/ethos/rigcheck.sh
EOF

Thanks to this script, crashed or hanging miners will be restarted fairly quickly and my rig’s pool-reported hashrate stopped dropping in such situations.

How to display crypto currency rates on macOS menu bar

Posted on by Jan

BitBar has been around for a while but I didn’t notice it until I wanted to display crypto currency rates (who isn’t into crypto these days :)) on my macOS menu bar. BitBar is as simple as it can get: it takes the output from a shell script and displays the result on the menu bar. What’s really cool about BitBar is the support for Base64 encoded images which get displayed as icons on the menu bar.

This is how my “crypto menu bar” looks like:

I just put all my BitBar shell scripts into ~/bitbar and that’s it.

Here are BitBar shell scripts for Monero, Ether and Bitcoin (against the €). Since the icons are just 16×16 they probably look crappy on a retina display and have to be replaced with larger icons.

Gigabyte Z97X-UD5H and USB 3.0 in macOS Sierra 11.12

Posted on by Jan

2000px-usb_icon-svgA commenter recently asked if I had any USB 3.0 related issues with my Gigabyte Z97X-UD5H equipped Hackintosh. Since every USB port was working out of the box I thought everything was fine. However, having a closer look at the USB section in macOS Sierra’s System Information revealed that none of the USB 3.0 ports were operating at USB 3.0 speeds, they were all capped at 480 Mb/sec.

Here’s how I was able to get USB 3.0 speed back:

  1. Inspired by https://github.com/RehabMan/OS-X-USB-Inject-All/blob/master/config_patches.plist I added the entire “Patches” section to my clover.plist. From the “KextsToPatch” section I only added the “Change 15 port limit to 20 in XHCI kext (9-series)”  related to macOS 11.12 and removed the disabled line.
  2. I downloaded https://bitbucket.org/RehabMan/os-x-fake-pci-id/downloads and copied FakePCIID.kext and FakePCIID_XHCIMux.kext to Clovers kext directory.
  3. I made sure XHCI mode was set to “Smart Auto” and both, XHCI and EHCI hand-off were enabled in the BIOS.

All front and back panel USB 3.0 ports are now reporting 3.0 speeds when connecting a 3.0 compatible device:

JSM578:
  Product ID:	0x0578
  Vendor ID:	0x152d  (JMicron Technology Corp.)
  Version:	2.03
  Serial Number:	DB123456789B
  Speed:	Up to 5 Gb/sec
  Manufacturer:	JMicron
  Location ID:	0x14f00000 / 20
  Current Available (mA):	900
  Current Required (mA):	896
  Extra Operating Current (mA):	396

Major kudos to RehabMan for providing these easy-to-use injector kexts!
Since injector kexts are not drivers and thus do not have to be signed, my Hackintosh still runs with maximum system integrity protection (SIP):

Jans-Mac:~ jan$ csrutil status
System Integrity Protection status: enabled.

Random delay for cron.daily, cron.weekly, cron.monthly

Posted on by Jan

cron-logoWouldn’t it be nice if cron’s daily, weekly and monthly jobs could be run with a slight offset? At least that’s what I thought when 20+ servers were hitting my backup infrastructure at once. The scripts in /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly are triggered directly from crontab at fixed times. Here’s what /etc/crontab looks like in Ubuntu Server 16.04:

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user	command
17 *	* * *	root    cd / && run-parts --report /etc/cron.hourly
25 6	* * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6	* * 7	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6	1 * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

I’ve found several tips which suggested to use a RANDOM_DELAY variable in crontab. Unfortunately, this variable doesn’t seem to be implemented in Debian/Ubuntu’s version of crontab at this time. I even checked the source code, there’s no RANDOM_DELAY variable to be found.

Here’s the solution I came up with. I’m using a combination of sleep and numrandom with a time range between 0 and 30 minutes.

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user	command
17 *	* * *	root    cd / && run-parts --report /etc/cron.hourly
25 6	* * *	root	sleep `numrandom /0..30/`m ; test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6	* * 7	root	sleep `numrandom /0..30/`m ; test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6	1 * *	root	sleep `numrandom /0..30/`m ; test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

In order to use the numrandom command, you have to apt-get -y install num-utils it first.

I didn’t delay the cron.hourly execution but the same sleep/numrandom combo could be used for it as well, just maybe replace the m (minutes) with s (seconds).

How to generate daily PowerDNS statistics reports

Posted on by Jan

daae8fe19d3ab224f3a104f987acd8bfPowerDNS has been powering authoritative DNS lookups to this web site for quite a while now. It’s such a remarkable piece of software. Here’s how to create a daily statistics report for PowerDNS.

  1. Put the lines below in /etc/cron.daily/powerdns-report:
    #!/bin/sh
    /usr/bin/curl -s localhost:8081 | mail -s "$(echo "PowerDNS Daily Report\nMIME-Version: 1.0\nContent-Type: text/html")" root
  2. Make sure the file is executable: chmod +x /etc/cron.daily/powerdns-report
  3. Enable the internal web server (defaults to 127.0.0.1:8081) in the PowerDNS configuration file:
    webserver=yes
  4. Restart PowerDNS
  5. Make sure all Mails to root are forwarded to your e-mail account (or replace the recipient in the powerdns-report script)

This is how the PowerDNS statistics report will look like:

PowerDNS 4.1.0
Uptime: 3.94 days
Queries/second, 1, 5, 10 minute averages: 0.402, 0.212, 0.186. Max queries/second: 1.17
Cache hitrate, 1, 5, 10 minute averages: 45.0%, 33.4%, 30.9%
Backend query cache hitrate, 1, 5, 10 minute averages: 55.1%, 41.7%, 38.3%
Backend query load, 1, 5, 10 minute averages: 0.343, 0.317, 0.32. Max queries/second: 2.37
Total queries: 52665. Question/answer latency: 1.25ms

Log Messages
gmysql Connection successful. Connected to database 'pdns' on 'localhost'.	422	27.8%
AXFR of domain 'nogo.com.' initiated by 231.91.120.148	70	4.6%
AXFR of domain 'nogo.com.' failed: 231.91.120.148 cannot request AXFR	70	4.6%
AXFR of domain 'nogo.com' denied: client IP 231.91.120.148 has no permission	70	4.6%
Rest:	527	34.7%
Total:	1519	100%

Queries for existing records, but for type we don't have
wopr.com./AAAA	2193	47.1%
wopr.me./AAAA	503	10.8%
drfalken.me./AAAA	482	10.4%
mail.wopr.net./AAAA	281	6.0%
wopr.net./AAAA	253	5.4%
Rest:	685	14.7%
Total:	4653	100%

Queries for non-existent records within existent domains
dn42-fr.wopr.me./A	359	24.9%
_sip._udp.sbc.sipmly.com.wopr.net./SRV	335	23.2%
_sip._udp.sip.ovh.fr.wopr.net./SRV	214	14.8%
dn42-us.wopr.me./A	75	5.2%
_adsp._domainkey.woprs.net./TXT	14	1.0%
Rest:	290	20.1%
Total:	1443	100%

UDP Queries Received
wopr.com./A	2307	23.1%
wopr.net./A	2227	22.3%
trick77.com./AAAA	702	7.0%
wheniskeynote.com./AAAA	578	5.8%
www.trick77.com./A	293	2.9%
Rest:	3135	31.4%
Total:	10000	100%

Queries that could not be answered due to backend errors
Total:	0	100%

Queries for domains that we are not authoritative for
nogo.com./SOA	190	84.4%
cpsc.gov./ANY	16	7.1%
dnsscan.shadowserver.org./A	4	1.8%
./ANY	3	1.3%
cpsc.gov./A	2	0.9%
defcon.org./ANY	2	0.9%
./NS	1	0.4%
4caa2d31.openresolvertest.net./A	1	0.4%
c.afekv.com./A	1	0.4%
doc.gov./ANY	1	0.4%
Rest:	4	1.8%
Total:	225	100%

Remote server IP addresses
137.24.55.110	289	2.9%
164.8.230.15	108	1.1%
8.91.147.135	106	1.1%
8.91.150.198	103	1.0%
28.1.240.64	100	1.0%
163.19.151.12	87	0.9%
15.16.90.42	84	0.8%
141.28.51.41	73	0.7%
169.16.124.226	70	0.7%
164.891.213.199	63	0.6%
Rest:	8917	89.2%
Total:	10000	100%

Remote hosts sending corrupt packets
219.116.116.12	1	50.0%
116.125.42.131	1	50.0%
Total:	2	100%

Remote hosts querying domains for which we are not auth
8.194.50.198	70	31.1%
8.194.47.135	68	30.2%
163.19.151.12	52	23.1%
185.59.233.131	15	6.7%
88.65.195.128	2	0.9%
64.32.96.66	2	0.9%
41.12.122.91	1	0.4%
45.20.24.195	1	0.4%
15.52.206.174	1	0.4%
89.146.222.158	1	0.4%
Rest:	12	5.3%
Total:	225	100%

Variables
corrupt-packets	2	Number of corrupt packets received
deferred-cache-inserts	1042	Amount of cache inserts that were deferred because of maintenance
deferred-cache-lookup	250	Amount of cache lookups that were deferred because of maintenance
dnsupdate-answers	0	DNS update packets successfully answered.
dnsupdate-changes	0	DNS update changes to records in total.
dnsupdate-queries	1	DNS update packets received.
dnsupdate-refused	1	DNS update packets that are refused.
incoming-notifications	0	NOTIFY packets received.
packetcache-hit	16206	
packetcache-miss	36486	
packetcache-size	1252	
query-cache-hit	52387	Number of hits on the query cache
query-cache-miss	97378	Number of misses on the query cache
rd-queries	212	Number of recursion desired questions
recursing-answers	0	Number of recursive answers sent out
recursing-questions	0	Number of questions sent to recursor
recursion-unanswered	0	Number of packets unanswered by configured recursor
security-status	0	Security status based on regular polling
servfail-packets	0	Number of times a server-failed packet was sent out
signatures	377	Number of DNSSEC signatures made
tcp-answers	497	Number of answers sent out over TCP
tcp-answers-bytes	434810	Total size of answers sent out over TCP
tcp-queries	299	Number of TCP queries received
tcp4-answers	485	Number of IPv4 answers sent out over TCP
tcp4-answers-bytes	432435	Total size of answers sent out over TCPv4
tcp4-queries	287	Number of IPv4 TCP queries received
tcp6-answers	12	Number of IPv6 answers sent out over TCP
tcp6-answers-bytes	2375	Total size of answers sent out over TCPv6
tcp6-queries	12	Number of IPv6 TCP queries received
timedout-packets	0	Number of packets which weren't answered within timeout set
udp-answers	52665	Number of answers sent out over UDP
udp-answers-bytes	4955023	Total size of answers sent out over UDP
udp-do-queries	42490	Number of UDP queries received with DO bit
udp-queries	52665	Number of UDP queries received
udp4-answers	40096	Number of IPv4 answers sent out over UDP
udp4-answers-bytes	3508260	Total size of answers sent out over UDPv4
udp4-queries	40096	Number of IPv4 UDP queries received
udp6-answers	12569	Number of IPv6 answers sent out over UDP
udp6-answers-bytes	1446763	Total size of answers sent out over UDPv6
udp6-queries	12569	Number of IPv6 UDP queries received
key-cache-size	12	Number of entries in the key cache
latency	1251	Average number of microseconds needed to answer a question
meta-cache-size	61	Number of entries in the metadata cache
qsize-q	0	Number of questions waiting for database attention
real-memory-usage	56033280	Actual unique use of memory in bytes (approx)
signature-cache-size	179	Number of entries in the signature cache
sys-msec	37296	Number of msec spent in system time
udp-in-errors	0	UDP 'in' errors
udp-noport-errors	45	UDP 'noport' errors
udp-recvbuf-errors	0	UDP 'recvbuf' errors
udp-sndbuf-errors	0	UDP 'sndbuf' errors
uptime	340341	Uptime of process in seconds
user-msec	57664	Number of msec spent in user time
© 2013 - 2016 PowerDNS.COM BV.

How to migrate a LVM-based KVM guest to another host

Posted on by Jan

kvm-logo_300dpiIn the past, I have been using the immensely useful virt-backup.pl script to migrate a LVM-based (raw volume) Linux KVM guest from one host to another. However, there is an even easier way to cold-migrate a KVM guest. This approach is particularly helpful if there’s not enough disk space on the host to create a gzipped backup of the logical volume using the virt-backup.pl script.

Here’s how it works:

  1. Use lvcreate to create the new logical volume on the destination host with the same size as the source logical volume. Use the lvdisplay command to find out the required size.
  2. virsh shutdown the source KVM guest
  3. On the source host: screen dd if=/dev/vg_ssd/lv_vm_wopr | pv | ssh -C root@desthost dd of=/dev/vg0/lv_vm_wopr
  4. Wait until finished

I’m using the screen command so it will continue running in the background once I close the ssh session on the source host. Use CTRL-A-D to background a screen session and screen -dr to bring it back up.

Using ssh makes sure the entire transfer is encrypted. The -C parameter makes sure the content will be compressed which may speed up the transfer considerably (or not, on a slow CPU). Obviously, the new KVM guest has to be virsh define‘d on the destination based on the virsh dumpxml configuration data from the source host.

Intel Gigabit CT kext for macOS Sierra 10.12

Posted on by Jan

macos-sierraThe Intel Gigabit CT Desktop ethernet PCI adapter is still one of the fastest and most robust NICs for the Hackintosh. This did not change with macOS Sierra 10.12. I’m still using the IONetworkingFamilyInjector.kext in Clover’s kext folder to override the compatibility list in Apple’s own Intel82574L.kext. However, while the installation of macOS Sierra went smoothly, I lost all network connectivity after installing Sierra. A quick look at the network kernel extensions revealed that Apple changed the driver identifier of the Intel82574L.kext, rendering the injector useless. After changing the identifier in the injector and a reboot, network connectivity was back again.

The patched injector kext is available for download here: IONetworkingFamilyInjector.kext_.macos-sierra.zip. The kext injector has to be placed into the EFI/CLOVER/kexts/10.12 folder.

intel gigabit desktop ct

The Hackintosh is still running in full protected mode (if enabled in Clover):
$ csrutil status
System Integrity Protection status: enabled.

A permanent solution?

While writing this post, I stumbled upon an alternative solution, which seems to be permanent. However, it requires flashing the Intel NIC and changing it’s device ID property. Check out this post on InsanelyMac. I’m going to try this approach in the near future since it would reduce the number of kexts in my Hackintosh rig to just one (only FakeSMC).

Installing Ubuntu Server 16.04 on PC Engines APU or APU2

Posted on by Jan

Most people use PC Engines APU series (APU1D4, APU2C4) system boards for pfSense firewalls (pfSense is awesome!). However, the Ubuntu Server x86-64 version runs on these boards very well too which can turn them into a lightweight, portable Plex Media Server for instance. The APU series doesn’t have a video port, that’s why the Ubuntu Server 16.04 image requires some modifications in order to use the serial port for console output instead. Since the Ubuntu image is using a read-only CD-ROM filesystem, I’m using UNetbootin to create a bootable Live USB drive which lets me modify files. While UNetbootin is available on Linux and MacOS too, only the Windows version gave me consistent results after formatting the USB drive to FAT32 file format. YMMW, but if you get weird bootloader errors, try formatting/creating the bootable drive on Windows.

To access the APU’s serial port, a RS-232 DB9 null-modem to USB interface is required and some software to connect to it. I’m using a Prolific PL-2303 based interface and minicom on Linux or Serial when I’m on my Mac.

Once the Live USB drive has been successfully prepared by UNetbootin, the following files have to be modified in order to send the console output over the serial port:

In /isolinux/isolinux.cfg, insert the following lines at the top:

serial 0 115200
console 0

In /isolinux/txt.cfg, the replace the first occurrence of the append keyword (in the “install” section) with:

append file=/cdrom/preseed/ubuntu-server.seed vga=off initrd=/install/initrd.gz -- console=ttyS0,115200n8 -

In /syslinux.cfg, insert the following lines at the top:

serial 0 115200
console 0

Again in /syslinux.cfg, replace the first occurrence of the append keyword (in the “unetbootindefault” section) with:

append initrd=/ubninit vga=off console=ttyS0,115200n8 --

Using the serial cable you should now be able to install Ubuntu Server 16.04 on the APU:

apu-ubuntu-serial-console-1apu-ubuntu-serial-console-2

During the installation:

  • Make sure the APU is connected to a router. While configuring the network, always keep in mind that the rightmost network port ist the first port (eth0 or enp1s0)
  • Make sure to include “OpenSSH server” when choosing software to install

Most likely, there won’t be any visible console output (i.e. a login prompt) after the first reboot because the installer didn’t add the necessary parameters to GRUB_CMDLINE_LINUX. This is where the SSH daemon comes in handy (-:

To fix this, use SSH to login to the server and modify /etc/default/grub to include the following line:

GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,115200n8"

Run update-grub , reboot the APU and eventually there should be a login prompt:

apu-ubuntu-serial-console-3

Adding a DS3231 Real Time Clock to the Raspberry Pi 3

Posted on by Jan

ds3231-rtcSince the Raspberry Pi 3 doesn’t come with a battery-powered real time clock, it will only show the correct time once it has Internet connectivity (thanks to the NTP daemon). If the Raspberry Pi 3 is not connected to the Internet, you might want to add a hardware clock to set the current date. Here’s how to add a DS3231 real time clock GPIO module to the Raspberry Pi 3 in Raspbian Jessy Lite:

  1. Get a DS3231 real time clock module and install it on the GPIO header of the Raspberry Pi 3 on pin 1
  2. Add the following line at the end of /boot/config.txt in Raspbian Jessy: 
    dtoverlay=i2c-rtc,ds3231
  3. We don’t need fake-hwclock anymore: 
    apt-get purge fake-hwclock
  4. Check/set the current system time and write the system time to the RTC module using: 
    hwclock -w
  5. Set the correct time zone using: 
    dpkg-reconfigure tzdata
  6. Edit /etc/rc.local and add the hwclock command above the line that says “exit 0”: 
    /sbin/hwclock -s
  7. The /etc/init.d/hwclock.sh shell scripts tends to corrupt this RTC clock module. In my case, the RTC clock was set to 2066/01/01 after every reboot. To prevent this from happening, edit /etc/default/hwclock and set HWCLOCKACCESS to no: 
    HWCLOCKACCESS=no
  8. Reboot
  9. Done! Raspbian will now set the time from the RTC clock during boot even if there is no Internet connectivity available.
  10. If RTC corruption is still happening, you may have to get rid of the NTP daemon as well using: 
    apt-get purge ntp
apt-get install ntpdate
  11. After the NTP daemon has been removed, you can still sync the system clock using ntpdate-debian which you might add to /etc/rc.local as well (after the hwclock command though) – just in case there is an Internet connection available during boot. And/or add it to /etc/cron.daily for example.

Raspbian Jessy Lite will detect the DS3231 real time clock module automatically (as a DS1307 module but nevermind), there’s no need to whitelist or blacklist any I2C modules. There’s no need to run the i2cdetect command from the i2c-tools package. Once the clock module is detected, this line should be visible using dmesg:

# dmesg | grep rtc
[    6.640799] rtc-ds1307 1-0068: rtc core: registered ds3231 as rtc0

Check /proc/driver/rtc for more data on the RTC:

# cat /proc/driver/rtc
rtc_time : 19:26:18
rtc_date : 2016-03-25
alrm_time : 00:00:00
alrm_date : 1970-01-01
alarm_IRQ : no
alrm_pending : no
update IRQ enabled : no
periodic IRQ enabled : no
periodic IRQ frequency : 1
max user IRQ frequency : 64
24hr : yes

Query status information from Huawei’s HiLink 3G/LTE modems

Posted on by Jan

While Huawei provides status information for its HiLink modems via a web page, this is hardly useful when using the modem on a headless Linux server. I just published a small Python-based command-line tool on Github which displays some useful information about the modem’s status.

root@wopr~#: python ./hstatus.py
Huawei E3372 LTE Modem (IMEI: 121032526613216)
 Hardware version: CL1D3271AM Ver.A
 Software version: 22.286.53.01.161
 Web UI version: 16.100.05.00.03-Mod1.4
 Serial: L8FDW11512114431
 MAC address (modem): 00:0D:87:12:1C:1D
 Connection status: Connected
   Network type: UMTS (3G)
   Signal level: ▁▃▄▆█
   Roaming: Enabled
   Modem WAN IP address: 10.197.75.231
   Public IP address: 185.13.106.181
   DNS IP addresses: 212.113.0.5, 66.28.0.62
   Network operator: Swisscom
   Connected for: 03:15:15 (hh:mm:ss)
   Downloaded: 615.17 KB
   Uploaded: 258.69 KB
 Total downloaded: 14.69 MB
 Total uploaded: 1.34 MB
 Unread SMS: 1

The tool has been tested on a Huawei E3276 and a E3372 modem. For the newer E3372 modem I had to add some code to supply a RequestVerificationToken in the HTTP header.

Feel free to send a pull request on Github with your own tweaks!

The repository is available here: github.com/trick77/huawei-hilink-status