Home
RSS 2.0 news feed
News Archive
Search

Public Sources
  Mercurial Repositories

Git repositories
  logproc
  ausadmin-web
  get-cert
  fat-set-uuid
  djbdns
  LS30
  xbee-controller

Projects
  Electronics
  Arduino RFID Door Lock
  XBee network controller
  Hardware
  LS30 Burglar Alarm
  Online store
  Software
  logrun
  math-trainer
  mp3cd-tools
  fakepop
  aunic-modify
  ausadmin
  sms-notify
  apt-cacher
  video-capture
  wav-fixer
  misc

TRS-80 software I wrote
  System-80 Gallery
  Zeta BBS
  Utilities
  Printer Utils
  Patches
  Operating Systems
  Library of C
  Languages
  Include files
  Hardware
  Games
  FORTRAN Programs
  File Utils
  Disk Utils
  Comms Programs
  CMD File Utils
  BASIC Programs
  Uni Assignments
  Fidonet

My PGP Key

Hats I wear
  aus.* newsgroups
  www.news-admin.org

Philosophy
  New Technology
  Ethical Internet Advertising
  Digital Freedom
  Anti-spam
  Age of plenty
  Ditch Windows

Technical
  Big Server
  Spam

Unclassified
  Zeta Internet
  Links

Your donation gives me time
to develop more cool free stuff

Stop the Australian (Anti)Vaccination Network

Valid XHTML 1.0

Valid CSS!

ipv6 ready

Nick Andrew News

2010-04-25 - No sound after upgrading to Ubuntu 9.10 (Karmic Koala)

This affected my EEE 901. The symptoms were:

  • No sound output on the laptop's built-in device (snd-hda-intel)
  • When I plugged in a USB sound device (snd-usb-audio), it also didn't produce any sound
  • Pulseaudio showed only a "dummy" output device. No hardware devices shown
  • "cat /proc/asound/cards" showed the sound cards were recognised by the kernel

The problem turned out to be with my $HOME/.asoundrc and $HOME/.asoundrc.asoundconf files. I found this:

  • "alsamixer" failed to run, complaining of problems with the above files

Solution:

  • I removed .asoundrc and .asoundrc.asoundconf
  • I killed and restarted pulseaudio
    • pulseaudio -k
    • pulseaudio --start

Then sound worked. I assume that pulseaudio couldn't get the ALSA device information due to the problems with the config file(s). It was probably only .asoundrc.asoundconf with a problem but I had nothing special in the other file so I removed that too.

2010-04-16 - GPT, Grub and RAID

GPT is the Generalised Partition Table which is a new standard for partitioning a hard disk. Its advantages over the MSDOS partition table used since the 1980s are:

  • larger devices (over 2T)
  • more partitions per device
  • extended set of partition types

Grub 2 (versions 1.96 and later) can boot disks which use GPT, but there are some considerations which are detailed below.

What's the MSDOS partition table?

The MSDOS partition table was invented when disk drive capacities were around 5-20 megabytes and sectors were addressed using a (C,H,S) 3-tuple: Cylinder, Head and Sector numbers. This addressing scheme has been obsolete for a very long time now, and has been kludged several times to cope. Firstly disk drive capacities increased enormously - and we started to use geometries with large numbers of heads (63) and sectors (255) to cope. Secondly disk drives no longer have a fixed geometry: although the number of heads is fixed (up to twice the number of physical platters the drive has), there is a varying number of sectors per track, as more data is stored on the outside of each platter. Disk drives store an approximately constant amount of data per square millimetre of disk surface. Thirdly, modern drives provide internal error correction by relocating sectors away from disk areas with physical defects.

The MSDOS partition table uses CHS and also provides an LBA representation of each partition. LBA is a linear numbering of sectors independent of the physical drive geometry in which sectors are numbered starting from zero and going to a very high number (depending on device listed capacity) and we have relied on this for some years now - as the size of devices exceeded the capacity of CHS to describe, operating systems started ignoring the CHS values and using only LBA. But LBA's capacity is about to run out; it has a hard limit of 2 TB (1 terabyte = 2^40 bytes) and we will soon surpass that capacity.

What's GPT?

GPT uses an LBA system with a much larger capacity. It has provision for more partitions per device (although in my experience, with larger device sizes we make larger partitions, not more partitions). MSDOS was limited to 4 so-called "primary" partitions and one of these could be used as a pointer to further "extended" partitions. It was a kludge, and GPT eliminates this kludge.

MSDOS also used a single byte to describe the contents of each partition. This was troublesome as almost all of the 256 possibilities have been used at various times (the linux fdisk utility contains a list). So GPT extends that with UUID based partition types - an almost limitless set.

Linux supports GPT partition tables (also called disk labels). A tool called 'gdisk' can create and edit them, and other partitioning tools have varying levels of support.

Grub 2 can boot disks partitioned with GPT. But there are some interactions between Grub, GPT and RAID, which is the reason for the existence of this article.

GPT, Grub2 and RAID considerations

  • GPT partitions can be made with 'gdisk'. Some of the tools are still immature.

  • Grub2 can boot GPT partitioned disks, but it needs a so-called "BIOS Boot Partition" of at least 32 Mbytes. That's where Grub2 stores some of its low level boot code.

  • Grub2 cannot boot off a RAID partition which uses version 1.0, 1.1 or 1.2 metadata. The RAID must have been created using version 0.90 metadata.

4K sector disks

Storage manufacturers are now shipping 4K sector disks. Why 4K not the previous standard of 512 bytes? 4K sectors are more efficient for such large devices: the drive can pick up more data with each read operation; the number of physical sectors on the device is reduced by 8 times (which reduces the size of relocation tables, for instance); 4K matches the page size used in the x86 architecture. There are a lot of good reasons why.

The first Western Digital 4K drives were released with a compatibility mode enabled in which the drive simulates 512 byte sectors. This can affect performance quite a lot. The installer should ensure that all partitions are a multiple of 8 sectors in size. This affects GPT partitioning as well as MSDOS. It's a bit easier to ensure this in GPT because there's no geometry baggage, unlike MSDOS.

2010-04-09 - LS-30 GSM interface

The LS-30 can notify an alert via its internal PSTN interface. There is also an optional GSM interface (pictured), which takes a SIM card and can be used instead of or in addition to the PSTN notification.

LS-30 optional GSM interface

Alerts are notified via SMS message at first, and this is followed by a voice call. Five numbers can be configured for GSM. The SMS message looks like this:

Burglar       01-03 19:19 2010/04/02

The "01-03" refers to the device which triggered.

My monitoring software can detect a Burglary message and SMS me directly. I get the impression that it takes a couple of minutes for the LS-30 to get around to sending an SMS. It may be that the LS-30 takes time to check the PSTN line for a dial tone. The LS-30 seems to try to alert via PSTN first, and tries GSM second.

My monitoring system sends an SMS of this form:

Burglary at Fri Apr 9 10:48:50 EST 2010 zone 500 group 00

The zone and group numbers would correspond (on a real burglary) to the "01-03" shown above.

2010-04-04 - Reverse-Engineering the LS-30

In February 2010, after long investigation, I purchased an LS-30 Alarm System From Securepro Security. Having moved recently, I needed an alarm system. I'd had many years experience with the Solution 16 (PDF) range of alarms, originally designed and manufactured by EDM (a Zeta Internet customer) and now owned by Bosch. But I thought perhaps technology has improved in the meantime, and I wanted something more powerful, configurable, ... more geeky. That's the LS-30.

View of LS-30 from the front

The LS-30 is an alarm system designed for self-installation in a home or business. It communicates wirelessly with its devices, which include:

  • PIR (Passive Infra-Red) burglar sensors
  • Remote control key fobs
  • Solar powered outdoors siren
  • Magnetic reed switch door sensors
  • Smoke detectors
  • And more.

It is very configurable, and according to the vendor has "more features than any other system available". Except perhaps for very expensive commercial systems. What appealed to me though, was the ethernet interface - I can plug it into my network and (hopefully) configure it remotely.

I have begun to reverse-engineer the LS-30 communications protocol. Furthermore, I have released my code under the GNU General Public License (Version 3) so others in the community can benefit from this effort (and help me finish the job). See the links in the left column to my GitHub project.

2009-06-13 - dnscache fixed

I was able to solve the problem with dnscache. Basically, before sending each new request to a nameserver (for a particular query), dnscache would close the socket used for the last request. So any late response would not reach dnscache.

The code flow was roughly this:

+->  close socket
|    open socket()
|    bind() local end of socket to random port
|    connect() socket to next destination
|    send() request packet
+-<  poll with timeout for a response

The fix turned out to be fairly simple. Open a socket before sending the first request packet for a particular query, do not connect it to the destination, and use sendto() to specify each destination instead of send(). So the fixed code flow now looks like this:

     open socket()
     bind() local end of socket to random port
+->  sendto() request packet to next destination
|
+-<  poll with timeout for a response
     close() socket

The improvement on performance is extreme. I'm testing on a virtual machine using the 'netem' module to artificially create network latency of 5000ms. Before patching, dnscache took 310 seconds to lookup 'A www.telstra.net' - much longer than I had calculated in the previous post, because in fact dnscache had to send more requests than expected due to missing nameserver glue, perhaps for the net domain. After patching, dnscache was able to resolve 'www.telstra.net' in only 16 seconds. It sends 6 queries then receives the response to the first, and closes the socket so subsequent responses are ignored (they're not needed anyway).

You might think this patch is not necessary because 5000ms network latency is an extreme test and most internet hosts will have much lower latency (it's usually a leaf-node problem) but I experienced it on a modern HSDPA network. Also I have seen saturated dialup and ISDN connections with very high latencies (over 3000ms for ISDN). Also dnscache's timeouts start at 1 second so performance will start to degrade as soon as average request latency exceeds 1000ms. The more nameservers a domain has, the longer it will take to lookup because dnscache sends a request to every nameserver with a 1-second timeout before changing to 3 seconds (then 11, then 45).

I've made the patch available on github.com in a new repository I made called djbdns. The url is:

http://github.com/nickandrew/djbdns/tree/high-latency-patch

This repository also contains all public releases of djbdns to date: version 1.01 through 1.05. DJB has put djbdns into the public domain so it is OK for me to do this. There are also some patches written by other people, linked from tinydns.org. Feel free to clone, fork and submit further patches to this djbdns repository on github.

2009-06-08 - dnscache fails spectacularly on high latency connections

Like most people I use dnscache extensively for name resolution within my network. All this worked fine until recently when my DSL link broke (with the telephone line) and I had to use HSDPA for external connectivity.

My HSDPA provider is supposed to provide a maximum speed of at least 1.8 Mbit/sec - of course this can be lower due to network congestion, poor signal strength, low capacity network links and so on. I don't think I have ever received the full 1800 kbit/sec from my provider, maybe nothing over 300 kbit/sec in fact.

Anyway this particular outage was particularly troublesome because I was getting only 50 kbit/sec through HSDPA ... slower than a dialup modem. It was slow in another respect too, the packet round-trip time was between 5 and 7 seconds. That's over 200 times more than a typical RTT on DSL of 25 ms.

I don't know what caused the extremely high latency, but I do know what its effect was. Dnscache failed almost completely. It would send out requests to nameservers on the internet, and not receiving any response in a reasonable time, would go on to try the next nameserver, and the next, and so on. The responses came back at some later time ... after dnscache had given up on the request (and dnscache would ignore the response). So the net effect was a storm of DNS packets sent and received, as well as ICMP port-unreachable packets when responses were received after dnscache had stopped listening.

Now that the DSL is working I am testing dnscache (from djbdns 1.05) to see the exact nature of this problem and if it can be fixed. I am using Linux's Traffic Control subsystem (see lartc.org for documentation) and specifically the 'netem' queue discipline module to simulate various amounts of latency to a virtual host.

I setup the variable latency using this script:

#!/bin/bash
#
#  Setup network delay for testing dnscache on high latency links
#  Outbound traffic on eth0 is delayed by several seconds if
#    - ip dst is 192.168.1.75

set -x

tc qdisc del dev eth0 root 2>/dev/null

# Setup root class and base rates
tc qdisc add dev eth0 root handle 1: htb default 99

tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit
# all traffic
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 100mbit
# delayed traffic
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 2mbit
# default
tc class add dev eth0 parent 1:1 classid 1:99 htb rate 3500kbit

tc qdisc add dev eth0 parent 1:10 handle 10: sfq
tc qdisc add dev eth0 parent 1:11 handle 11: netem delay 5000ms
tc qdisc add dev eth0 parent 1:99 handle 99: sfq

# Move selected traffic into 1:11
tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip dst 192.168.1.75 flowid 1:11

This script is run on my gateway and it uses the HTB queue discipline on device eth0 and some classes beneath that to filter outbound packets on that interface. Packets sent to 192.168.1.75 are delayed (by the 'netem' qdisc) by exactly 5000 msec, which is more than enough time to give dnscache a headache.

Next up I hacked dnscache to (a) run as an ordinary user, (b) listen on port 5300, and (c) show me what it is doing. Dnscache implements exponential timeouts on sent requests for 1, 3, 11 and 45 seconds. I don't know why djb chose those particular numbers. I started the server and sent a single request to lookup an 'A' record for telstra.net:

dig -p 5300 @127.0.0.1 A telstra.net +tries=1

This is what dnscache output at first:

$ ./dnscache
starting
query 1 7f000001:e4d2:6114 1 telstra.net.
tx 0 1 telstra.net. . 803f0235 c03a801e c0249411 c6290004 c707532a 80080a5a c0702404 c0cbe60a ca0c1b21 c0e44fc9 c00505f1 c021040c c1000e81
dns_transmit to 128.63.2.53, timeout set to 1
dns_transmit to 192.58.128.30, timeout set to 1
dns_transmit to 192.36.148.17, timeout set to 1
dns_transmit to 198.41.0.4, timeout set to 1
dns_transmit to 199.7.83.42, timeout set to 1
dns_transmit to 128.8.10.90, timeout set to 1
dns_transmit to 192.112.36.4, timeout set to 1
dns_transmit to 192.203.230.10, timeout set to 1
dns_transmit to 202.12.27.33, timeout set to 1
dns_transmit to 192.228.79.201, timeout set to 1
dns_transmit to 192.5.5.241, timeout set to 1
dns_transmit to 192.33.4.12, timeout set to 1
dns_transmit to 193.0.14.129, timeout set to 1

What seems to be happening here is that dnscache is looking up the root nameservers ('.') for 'telstra.net'. The hex numbers are the IPv4 addresses of each root nameserver. Dnscache tries them in order (at least, in the same order as they appear on the 'tx' line). There are 13 root nameservers and these requests appear to be issued once per second - so this process has taken 13 seconds so far. With a 5000 msec delay on the interface, 8 of those 13 requests have been replied-to, but dnscache apparently stops listening for a response as soon as its timeout expires (1 second here) and it sends the next request.

Continuing on:

dns_transmit to 128.63.2.53, timeout set to 3
dns_transmit to 192.58.128.30, timeout set to 3
dns_transmit to 192.36.148.17, timeout set to 3
dns_transmit to 198.41.0.4, timeout set to 3
dns_transmit to 199.7.83.42, timeout set to 3
dns_transmit to 128.8.10.90, timeout set to 3
dns_transmit to 192.112.36.4, timeout set to 3
dns_transmit to 192.203.230.10, timeout set to 3
dns_transmit to 202.12.27.33, timeout set to 3
dns_transmit to 192.228.79.201, timeout set to 3
dns_transmit to 192.5.5.241, timeout set to 3
dns_transmit to 192.33.4.12, timeout set to 3
dns_transmit to 193.0.14.129, timeout set to 3

Dnscache sends to the same set of 13 nameservers, but with a 3 second timeout on each. That takes 39 seconds (for a total time spent so far of 52 seconds, and we still don't know what are the nameservers for telstra.net). Continuing:

dns_transmit to 128.63.2.53, timeout set to 11
rr 803f0235 172800 1 a.gtld-servers.net. c005061e
rr 803f0235 172800 1 b.gtld-servers.net. c0210e1e
rr 803f0235 172800 1 c.gtld-servers.net. c01a5c1e
rr 803f0235 172800 1 d.gtld-servers.net. c01f501e
rr 803f0235 172800 1 e.gtld-servers.net. c00c5e1e
rr 803f0235 172800 1 f.gtld-servers.net. c023331e
rr 803f0235 172800 1 g.gtld-servers.net. c02a5d1e
rr 803f0235 172800 1 h.gtld-servers.net. c036701e
rr 803f0235 172800 1 i.gtld-servers.net. c02bac1e
rr 803f0235 172800 1 j.gtld-servers.net. c0304f1e
rr 803f0235 172800 1 k.gtld-servers.net. c034b21e
rr 803f0235 172800 1 l.gtld-servers.net. c029a21e
rr 803f0235 172800 1 m.gtld-servers.net. c037531e
rr 803f0235 172800 ns net. a.gtld-servers.net.
rr 803f0235 172800 ns net. b.gtld-servers.net.
rr 803f0235 172800 ns net. c.gtld-servers.net.
rr 803f0235 172800 ns net. d.gtld-servers.net.
rr 803f0235 172800 ns net. e.gtld-servers.net.
rr 803f0235 172800 ns net. f.gtld-servers.net.
rr 803f0235 172800 ns net. g.gtld-servers.net.
rr 803f0235 172800 ns net. h.gtld-servers.net.
rr 803f0235 172800 ns net. i.gtld-servers.net.
rr 803f0235 172800 ns net. j.gtld-servers.net.
rr 803f0235 172800 ns net. k.gtld-servers.net.
rr 803f0235 172800 ns net. l.gtld-servers.net.
rr 803f0235 172800 ns net. m.gtld-servers.net.
rr 803f0235 172800 28 a.gtld-servers.net. 20010503a83e00000000000000020030
stats 1 945 1 0
cached 1 a.gtld-servers.net.
cached 1 b.gtld-servers.net.
cached 1 c.gtld-servers.net.
cached 1 d.gtld-servers.net.
cached 1 e.gtld-servers.net.
cached 1 f.gtld-servers.net.
cached 1 g.gtld-servers.net.
cached 1 h.gtld-servers.net.
cached 1 i.gtld-servers.net.
cached 1 j.gtld-servers.net.
cached 1 k.gtld-servers.net.
cached 1 l.gtld-servers.net.
cached 1 m.gtld-servers.net.

Dnscache has finally increased its timeout to 11 and after another 5 seconds (total time elapsed now 57 seconds) it receives a response. Now it knows the nameservers for the 'net' top-level domain. There are 13 of them and so it's going to take another 57 seconds before it learns the 4 nameservers for 'telstra.net', and then another 21 seconds to learn that there is actually no 'A' record for 'telstra.net'. That's 135 seconds total time. I don't know how long clients typically wait for a response but it's a lot less than that.

Clearly dnscache should implement two timeouts per request: one for sending a second request (to another nameserver) for the same information, and one to give up waiting for a response from the first request. The 2nd timeout should be much longer than the first.

If dnscache was modified to wait up to 10 seconds for a response but try each successive nameserver after 1 second, then it should be possible for dnscache to answer the query within 15 seconds, which is reasonable in this context. In these calculations I'm assuming that dnscache has no existing cache (other than the list of root nameservers) because that makes dnscache's behaviour predictable, and it shows us the worst case performance.

The next step for me is to work out if dnscache can have multiple outstanding requests to nameservers for the same client request. Obviously dnscache can handle multiple concurrent client requests and must query many different nameservers concurrently, but the question is whether it can query multiple nameservers for the same information, at more or less the same time, and use the first response that is received.

2008-08-25 - Configuring IPv6 using AARNet's free broker

You can obtain IPv6 connectivity in Australia using the AARNet IPv6 Migration Broker if your ISP does not already provide IPv6. As at August 2008, only Internode is known to provide consumer level IPv6 access.

For Australian users who cannot obtain IPv6 through your ISP, the AARNet IPv6 tunnel is probably the next lowest latency choice. These instructions describe setting up IPv6 with linux over a consumer DSL connection with a dynamic IPv4 address (i.e. every time you connect you get a different IP address) and NAT (Network Address Translation) on the DSL modem.

With IPv6 you can get a static address (well, 2^64 or more static addresses) and it's quite useful if you run a home network and want to provide access to devices within the network, from outside. That is, so long as the outside client has IPv6 (the chicken-and-egg problem).

Certain assumptions are made in this document - like installed software or an IPv6 enabled kernel. If these instructions don't work for you, drop me an email at nick at nick-andrew.net.

Preparation - Kernel and Software

Make sure your kernel has IPv6 support. Do "ifconfig eth0" and check for a line like this in the output:

          inet6 addr: fe80::212:34ff:fe56:789a/64 Scope:Link

You may need to load the "ipv6" module.

Check that you have the commands "ip6tables" and "ip".

Preparation - Firewall

It's good for security to setup the IPv6 firewall before you even start configuring it. Although you may have an IPv4 firewall already in place, that won't stop any IPv6 packets. I'll assume you want to make connections out, but block all connections in, which is the standard functionality you get with NAT, and useful for a client. For a server you will need to allow connections in for the server ports.

ip6tables setup

I use a shell script with a function alias, to set the firewall.
#!/bin/bash
#    Simple IPv6 firewall rules

doCmd6() {
        /sbin/ip6tables $*
        rc=$?
        if [ $rc != 0 ] ; then
                echo Error $rc doing ip6tables $*
        fi
}

doCmd6 -P INPUT DROP
doCmd6 -P FORWARD DROP

doCmd6 -F
doCmd6 -X

doCmd6 -N 6_log_drop
doCmd6 -N 6_icmp
doCmd6 -N 6_tcp

doCmd6 -A 6_log_drop -j LOG --log-prefix "ipv6:"
doCmd6 -A 6_log_drop -j DROP

#enable ssh in   doCmd6 -A 6_tcp -p tcp -d ::/0 --dport 22 -j ACCEPT
doCmd6 -A 6_tcp -j 6_log_drop

doCmd6 -F 6_icmp
doCmd6 -A 6_icmp -j ACCEPT


doCmd6 -A INPUT -i lo -j ACCEPT
doCmd6 -A INPUT -p ALL -m state --state ESTABLISHED,RELATED -j ACCEPT
doCmd6 -A INPUT -p icmpv6 -j 6_icmp
doCmd6 -A INPUT -p tcp -j 6_tcp
doCmd6 -A INPUT -j 6_log_drop

This set of rules will allow you to make outbound IPv6 connections and will block inbound except for ICMP (your address can be pinged) and you can uncomment a line to enable ssh connections in.

Preparation - DSL modem

Assuming your DSL modem has a firewall, you will need to enable two things:

  • Outbound UDP connections to port 3653
  • Protocol 41 inbound/outbound

UDP port 3653 is used by the AARNet tunnel broker. Protocol 41 is used by 6to4 tunneling, which I will describe now.

Alternate access - 6to4 tunnel

Although this document is about obtaining IPv6 through AARNet's migration broker, there's a quicker way to get started on IPv6 using your dynamic IPv4 address. You will need the "curl" command installed. Run this script:

#!/bin/bash
#  Sets up an ipv6 tunnel through the nearest public gateway.
#  Automatically learns current ipv4 address.

ipv4=$(curl -s http://www.whatismyip.com/automation/n09230945.asp)
ipv6=$(printf "2002:%02x%02x:%02x%02x::1" $(echo $ipv4 | tr "." " " ) )

echo ipv4 is $ipv4 and ipv6 is $ipv6

ifconfig sit0 up
ifconfig sit0 add $ipv6/16
route -A inet6 add 2000::/3 gw ::192.88.99.1 dev sit0

192.88.99.1 is an address which routes to the nearest public IPv6 gateway. Unfortunately for us Australians, that's probably in the USA. But it's easy to setup, as you can see.

This technique has some advantages and disadvantages. On the positive side, you have 2^80 addresses to use. That should be enough for anyone. On the negative side, these addresses are as dynamic as your IPv4 address because they are derived from it. Also, you may be able to make outbound connections but not accept inbound connections, depending on what functionality your DSL modem provides. My DSL modem allows me to open up protocol 41 traffic, but it doesn't provide a configuration option to forward all incoming protocol 41 connections to my linux box. So my box can only accept inbound connections while there's a working outbound connection (this is standard NAT functionality).

So 6to4 is a good technique to get started with IPv6 if you want to, say, browse ipv6 websites. But it's not so useful for servers.

Testing - IPv6 using commands

"ping6" and "telnet" are your friends.

$ ping6 ipv6.l.google.com
PING ipv6.l.google.com(2001:4860:0:2001::68) 56 data bytes
64 bytes from 2001:4860:0:2001::68: icmp_seq=1 ttl=52 time=308 ms
64 bytes from 2001:4860:0:2001::68: icmp_seq=2 ttl=52 time=335 ms
64 bytes from 2001:4860:0:2001::68: icmp_seq=3 ttl=52 time=321 ms
$ telnet luyer.net 80
Trying 2001:470:1f05:14d::1...
Connected to luyer.net.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.1 200 OK
Date: Mon, 25 Aug 2008 00:38:50 GMT
Server: Apache/2.2.3 (Debian) PHP/4.4.4-8+etch6 mod_ssl/2.2.3
OpenSSL/0.9.8c
[ ... etc ... ]

You can also install the netcat6 package (the command is "nc6") to connect out or listen for incoming connections. To test if your web browser is using IPv6 correctly, try these sites:

  • www.kame.net - on IPv6 you will see a swimming turtle, on IPv4 the turtle will not move
  • luyer.net and it will tell you if you connected through IPv4 or IPv6
  • www.sixxs.net will show your originating address and protocol at the bottom of the page.

Registering with AARNet

You'll need to create a username on the AARNet IPv6 Migration Broker and then associate that username with a tunnel.

  1. Create User Account
  2. Request Tunnel

The "Request Tunnel" CGI hangs for me, I don't know why. I assume it created a tunnel correctly because I'm getting one.

Installation of TSPC

Install the "tspc" package. I'm using Debian, so it's just "apt-get install tspc".

When Debian installs tspc, it automatically configures it to obtain an anonymous IPv6 tunnel from freenet6.net. Although that's great and easy, it's not what we need in Australia. So after installation, stop the daemon with "/etc/init.d/tspc stop".

Configuring the AARNet tunnel broker

To be continued ... no time to write the rest of the document today.

In brief, you need to use the v6udpv4 method and all traffic will be tunneled through a UDP connection. 30 second keepalive messages (I believe it's an ipv6 ping?) will keep NAT working. Your config file should look like this:

auth_method=any
userid=YOURUSERID
passwd=YOURPASSWORD

server=broker.aarnet.net.au
tunnel_mode=v6udpv4
host_type=router
prefixlen=64

client_v4=auto
if_prefix=eth0
if_tunnel_v6udpv4=tun
if_tunnel_v6v4=sit1

keepalive_interval=30
keepalive=yes

proxy_client=no
retry_delay=30
template=setup

Note that AARNet assigns a /56 even though this config asks for a /64.

Monitoring the IPv6 tunnel

AARNet's broker seems a bit flaky. It sometimes ignores the UDP packets so you may need to start the daemon a few times before it connects properly. As I mentioned, the CGI to request a tunnel hangs for me. If it doesn't hang, it may return a shell script, which you can ignore.

It's possible you don't even need to use the "Request Tunnel" CGI, maybe it will assign a permanent tunnel when tspc connects for the first time.

Also AARNet reboot the server at 03:00 every morning. The tspc daemon will try to reconnect if the tunnel fails, but perhaps not forever.

In any case, you want IPv6 to be always available, which means monitoring it. If the tspc daemon stops, you will need to restart it manually. I wrote this simple perl script which just logs whenever IPv6 stops working. It doesn't monitor the process or interface at all (you'd want to monitor the process itself if you change the script to restart a failed process).

#!/usr/bin/perl -w
#       @(#) $Id$
#       vim:sw=4:ts=4:

use Date::Format qw(time2str);
use Getopt::Std qw(getopts);

use vars qw($opt_h);

$| = 1;
getopts('h:');

$opt_h || die "Need option -h hostname";

# Initial and current state
my $state = 'down';

while (1) {
        my $now = time();
        my $newstate = getPingState($opt_h);

        if ($newstate ne $state) {
                open(OF, ">>ipv6-state.log");
                my $ts = time2str('%Y-%m-%d %T', $now);
                print OF "$ts $opt_h $newstate\n";
                close(OF);

                $state = $newstate;
        }

        my $to_wait = 60 - ($now % 60);
        print '.';
        sleep($to_wait);
}

# NOTREACHED
exit(0);

# ------------------------------------------------------------------------
# ping6 a host, and figure out if it is up
# ------------------------------------------------------------------------

sub getPingState {
        my $host = shift;

        my $rc = system("ping6 -c 2 -q -W 8 $host >/dev/null");
        if ($rc == 0) {
                return 'up';
        }

        return 'down';
}

I just run it like this: "check-ipv6.pl ipv6.l.google.com" and it pings once a minute (on the minute) and logs whenever the calculated state of the IPv6 connectivity changes.

For production use you'd want to change the 'sleep' to sleep(60) since it's not friendly to google for many sites to ping in synchronisation (i.e. always at :00). Of course you could always ping6 to broker.aarnet.net.au instead ...

64 bytes from 2001:388:1:5001:2a0:a5ff:fe4b:ae3: icmp_seq=1 ttl=64 time=52.8 ms
64 bytes from 2001:388:1:5001:2a0:a5ff:fe4b:ae3: icmp_seq=2 ttl=64 time=51.6 ms
64 bytes from 2001:388:1:5001:2a0:a5ff:fe4b:ae3: icmp_seq=3 ttl=64 time=51.1 ms
2008-08-24 - Forcing NFS to use consistent port numbers

NFS servers don't play nice with iptables when the daemons use random port numbers (which is the default in debian).

To force consistent port numbers upon NFS (when using nfs-kernel-server), set the following in /etc/default/nfs-kernel-server. This will force mountd to use port 4002 for both TCP and UDP.

RPCMOUNTDOPTS="--port 4002"

For lockd, add the following into /etc/modules (you need to build lockd as a module for this) ...

lockd    nlm_udpport=33000 nlm_tcpport=42049

Next, you can force statd to use port 4000 and port 4001 by setting this in /etc/default/nfs-common:

STATDOPTS="--port 4000 --outgoing-port 4001"

Finally, add iptables rules:

iptables -F nfs_server
iptables -A nfs_server -p TCP --dport 2049 -j ACCEPT # nfs
iptables -A nfs_server -p TCP --dport 4000:4001 -j ACCEPT # statd
iptables -A nfs_server -p TCP --dport 4002 -j ACCEPT # mountd
iptables -A nfs_server -p UDP --destination-port 4002 -j ACCEPT # mountd
iptables -A nfs_server -p UDP --destination-port 33000 -j ACCEPT # lockd
iptables -A nfs_server -p TCP --dport 42049 -j ACCEPT # lockd

Be sure that only packets from client IP addresses are directed through the nfs_server table.

2008-05-19 - Scan and update ssh keys

Due to the recently exposed ssh key generation vulnerability, many ssh keys may need to be deleted and recreated. This tool may help with that.

What it does is find all the ssh keys existing on several hosts and where those keys appear in .ssh/authorized_keys files. It collates the results from all hosts and writes a directed graph for the 'graphviz' tool, which helps to visualise which keys are providing access to which users.

The scan tool works by connecting to each specified host using ssh and executing a host script. The host script examines the user's .ssh directory and sends a report to STDOUT, which is received by the scan tool. The host script can also iterate through /etc/passwd and examine every user's .ssh directory.

Use graphviz to visualise the graph. How to use: firstly make a file 'host-list' of all hosts you have access to, one line per hostname. Include any hosts you have root access to just as the hostname, and if you have only user access, use "user@hostname". You need to run the scan tool from a place which can ssh to all these hosts -- as root for all hostnames listed in your 'host-list' file which are not qualified with a username.

scan-ssh-keys.pl $(cat host-list) > output.dot
dot -Txlib output.dot

Use the mouse wheel to zoom in/out of the graph image. Use the mouse middle button to pan the image. You can also use 'dot' to write various output files such as JPEG or SVG.

Here is a sample graph produced by the scan tool. The graph resolution is deliberately reduced (to make the node labels unreadable) for security reasons. Also my full ssh key graph is somewhat larger than this :-)

Sample output from ssh key scan tool

Later on I expect to enhance the scan tool to assist with adding and deleting ssh keys.

To download from my Mercurial repository, see: nick-src-regen-ssh-keys.

2008-04-25 - Distributed SCM

I've become a recent convert to distributed SCM systems, particularly Mercurial and Git.

Use of a distributed SCM is a much more sensible way to release open source software than the traditional tarfile, or even having a public CVS or SVN repository. In particular:

  • It makes it easy for users of the software to feed back their local changes, which encourages more participation and development of the code
  • It helps users maintain their own local changes in an SCM
  • It helps users upgrade to newer releases (without wiping out their local changes, for instance).

Having a public CVS or SVN repository solves only the problem of giving users quick access to the latest version of the software. CVS and SVN don't help with feedback or maintenance of local changes. They can help with upgrades - but it's of little practical benefit when local changes can't be committed into the repository.

I was in the process of converting my CVS repositories to SVN (it takes a long time) when I discovered the virtues of Distributed SCMs. I'm using both Git and Mercurial. Git is perhaps more powerful but Mercurial is more user-friendly. So I have halted all conversion to SVN and I'll be converting remaining repositories to Mercurial. I'll be converting most of my SVN repositories over to Mercurial too. Ultimately the only code/data remaining in SVN will be that which I particularly need to be centralised (it might end up being none at all).

I have started to put some repositories of my public source code online already. The full list is available at: http://www.nick-andrew.net/hg and individual repositories which are currently online are:

develooper.com-src-qpsmtpd I use qpsmtpd on the mail servers here primarily for spam entrapment; for example to refuse emails addressed to nonexistent users at my domains. I get quite a lot of spam due to my long history with USENET. This repository contains the qpsmptd 0.26, 0.32 and 0.40 releases as downloaded from tarballs plus my local changes (mostly against 0.26 so far).
nick-pub-src-minix-ar To extract my old stuff (circa 1990) successfully, I need a V7-compatible 'ar' program. This is the original 'ar' program from Minix 1.x, modified to work on a modern machine (32-bit, with prototypes, etc).
nick-src-apt-cacher This is the original apt-cacher program (versions 0.1 through 0.4) before being picked up by Jon Oxer and packaged into Debian. Don't use this code; the repository is here merely for historical purposes.
nick-src-backup-cdr I wrote this system to do encrypted incremental backups of modified files - first to CDR, then to DVD. I need something to do size limited incremental backups since the total amount of backup is many times the capacity of a DVD. This system regulates the quantity of backup per day.
nick-src-direct This is a perl implementation of the 'direct' program I used in the TRS-80 days, to pack several files together into 2 (!) structured files, a 'DIR' and a 'DAT' file. Think of it as like a zipfile without any compression. Anyway when I restored my old TRS-80 software I needed to extract data from some of these structured file pairs so I wrote it in perl - a language which was not available back in the mid-1980s. It was trivial to do in perl, heh.
nick-src-distributed-filesystem This repository contains my 2003 ruminations on distributed filesystems. Useful if nothing else for the pointers to various designs people may have implemented.
nick-src-incremental-tar This C program replicates the function of Gnu TAR and optionally compresses its output on the fly. The point is that you can feed it with a (possibly infinite) list of files to be archived, and an output size limit, and it will ensure that its output size does not exceed the limit. It is useful for incremental backup - you give it a list of all your modified files, and a size (say 699 MBytes) and it will cram as many of those files in compressed form as can fit in 699 megs.
nick-src-misc-https-proxy This perl program listens on localhost port 80 and when it receives a connection, makes an SSL connection to a specified server on port 443 and forwards the data in both directions. This allows logging or debugging of an HTTPS data stream by using HTTP at the browser end.
nick-src-misc-logrun This cute little program reopens STDOUT and STDERR to files in a logging directory and runs the program. If the program exits with a non-zero code, the log files are moved to a 'bad' directory, otherwise they are moved to an 'ok' directory.
nick-src-misc-showdns This program does several DNS lookups on a specified domain name, and possibly filters the output. The aim is to quickly find the nameservers, web servers etc for a domain name and particularly to find those which are _not_ served by a specified nameserver.
nick-src-trs80 This is all my old TRS-80 source code in a single repository, rather than having to download many tarfiles. Again, it's for historical purposes. Unless you like Z80 assembler a lot.