Setting Up DKIM And SRS In Postfix

In my previous post Custom Domain E-mails With Postfix And Gmail: The Missing Tutorial, we set up a Postfix mail server on a custom domain that integrates seamlessly with Gmail.

However, the tutorial skipped two important security standards that will help prevent e-mails routed through our server from being marked as spam: DKIM and SRS. This article will show you how to add support for DKIM and SRS to a Postfix server.

Similar to the previous tutorial, we will assume a Ubuntu server in our examples.

Step 1: DKIM

DKIM, short for DomainKeys Identified Mail, is a mechanism for

  1. A sender e-mail program to sign an outgoing e-mail message, and

  2. A recipient e-mail program to verify said signature.

More concretely, let’s say Gmail receives an e-mail message from myserver.example.com. DKIM allows Gmail to verify that the e-mail was indeed sent by the designated e-mail server program on myserver.example.com, and not by, say, a virus running on myserver.example.com or a malicious user who happens to have access to myserver.example.com.

We will use OpenDKIM for this tutorial. To install OpenDKIM on Ubuntu:

$ sudo apt-get install opendkim opendkim-tools

/etc/opendkim.conf

Edit /etc/opendkim.conf to match the following:

# OpenDKIM config.

# Log to syslog
Syslog                  yes
SyslogSuccess           yes
LogWhy                  yes
# Required to use local socket with MTAs that access the socket as a non-
# privileged user (e.g. Postfix)
UMask                   002

Mode                    sv
PidFile                 /var/run/opendkim/opendkim.pid
UserID                  opendkim:opendkim
Socket                  inet:[email protected]

Canonicalization        relaxed/simple
SignatureAlgorithm      rsa-sha256

# Sign for example.com with key in /etc/opendkim.d/mail.private using
# selector 'mail' (e.g. mail._domainkey.example.com)
Domain                  example.com
KeyFile                 /etc/opendkim.d/mail.private
Selector                mail

ExternalIgnoreList      refile:/etc/opendkim.d/TrustedHosts
InternalHosts           refile:/etc/opendkim.d/TrustedHosts

You can check out a detailed explanation for the meaning of each option with man opendkim.conf.

/etc/opendkim.d/TrustedHosts

Create the directory /etc/opendkim.d and put the following in /etc/opendkim.d/TrustedHosts. This instructs the OpenDKIM server to sign e-mails delivered by any server matching these expressions (such as myserver_2.example.com).

127.0.0.1
::1
localhost
192.168.0.1/24

*.example.com

DKIM keys

Now, let’s generate our DKIM signing keys:

$ cd /etc/opendkim.d
$ sudo opendkim-genkey -s mail -d example.com

This will produce two files in /etc/opendkim.d: our private key, mail.private, and our public key, mail.txt.

We should make sure only OpenDKIM can read the private key, so that a malicious program or user on the same server won’t be able to forge our signature:

$ chmod 600 mail.private
$ chown opendkim:opendkim mail.private

DKIM public key DNS record

The next step is to publish our public key through DNS, so that any recipient e-mail program can verify our signature. If we look at our public key mail.txt generated in the previous step, it should look like something like this:

mail._domainkey IN      TXT     ( "v=DKIM1; k=rsa; "
          "p=<alphabetical soup>" )  ; ----- DKIM key mail for example.com

Create the following TXT DNS record for example.com (how to update DNS records will depend on the DNS hosting provider):

  • Name: mail._domainkey.example.com

  • Value: v=DKIM1; k=rsa; p=<alphabetical soup>

where <alphabetical soup> is the public key found in mail.txt after p=. Note that DNS records will take a while (depending on our provider, up to a day) to propagate.

OpenDKIM server

We can now start the OpenDKIM server with

$ sudo /etc/init.d/opendkim start

Postfix

The last step is to tell Postfix to use OpenDKIM to sign outgoing e-mail messages. Add the following to /etc/postfix/main.cf:

# Milter settings.
milter_protocol = 2
milter_default_action = accept
# OpenDKIM runs on port 12301.
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301

If you already have other milters configured (such as SpamAssassin), simply add inet:localhost:12301 to your existing smtpd_milters and non_smtpd_milters lines, prefixed by a comma.

Let’s now restart Postfix with the new configuration:

$ sudo postfix reload

…and we’re done!

Step 2: SRS

SRS, short for Sender Rewriting Scheme, is a standard for including forwarding / relay information in a forwarded / relayed e-mail message.

For example, suppose [email protected] sends an e-mail to [email protected], and our Postfix server on myserver.example.com forwards this e-mail to [email protected]. SRS allows our Postfix server on myserver.example.com to attach a virtual sticky note on the e-mail message explaining this situation to Gmail. Otherwise, Gmail might become suspicious of why myserver.example.com is producing messages that purport to come from hotmail.com, which spammers and phishers are wont to do.

We will use PostSRSd to implement SRS in our Postfix server. It works out of the box with Postfix and is a breeze to set up, but unfortunately is not included in the official Ubuntu / Debian package repositories.

OpenSRSd

Let’s build and install OpenSRSd from source:

# Dependencies.
$ sudo apt-get install unzip cmake

# Download and extract source code from GitHub.
$ cd /tmp
$ curl -L -o postsrsd.zip \
    https://github.com/roehling/postsrsd/archive/master.zip
$ unzip postsrsd.zip

# Build and install.
$ cd postsrsd-master
$ mkdir build
$ cd build
$ cmake -DCMAKE_INSTALL_PREFIX=/usr ../
$ make
$ sudo make install

The default config provided by PostSRSd (/etc/default/postsrsd) will pretty much work out of the box for our case.

The install script will also conveniently install an Upstart script for PostSRSd. Let’s start it now:

$ sudo service postsrsd start

Postfix

Finally, we configure Postfix to use PostSRSd. Add the following to /etc/postfix/main.cf:

# PostSRSd settings.
sender_canonical_maps = tcp:localhost:10001
sender_canonical_classes = envelope_sender
recipient_canonical_maps = tcp:localhost:10002
recipient_canonical_classes= envelope_recipient,header_recipient

And restart Postfix with the new configuration:

$ sudo postfix reload

That’s it!

Custom Domain E-mails With Postfix and Gmail: The Missing Tutorial

Custom domain e-mail addresses, like [email protected], are cool and professional-looking. Would you want to e-mail potential clients as [email protected]?

On the other hand, Gmail has great reliability, speed, spam filtration, and app support. Wouldn’t it be great if we could send and receive e-mail as [email protected], but right from the comfort of our Gmail account?

Today, if we want to use Gmail to send and receive e-mails on a custom domain, we have a couple of options:

  • Google Apps for Work is Google’s own offering. It’s simple to set up and manage, but it’s pretty pricey at $5/mo per user, especially if we don’t need the bundled collaborative features like Google Docs and Calendar.

  • Third-party services could get the job done at a lower price point; for example, Pobox offers plans starting at $20/yr. The downside is that we will have to trust another organization with our e-mail.

  • Set up our own e-mail forwarding server. It’s fairly easy (if we follow this guide :) and if we already have shared / dedicated hosting or a VPS, this is basically free and gives us total control over our e-mail.

If you want to host custom domain e-mails in Gmail without paying for Google Apps or a third-party service, this tutorial is for you. It will show you how to quickly set up you own e-mail forwarding / relay server, and how to integrate it seamlessly with Gmail.

Prerequisites

For this tutorial, we’ll assume we have access to:

  • VPS or shared / dedicated hosting, with a dedicated IP address and capable of listening on ports 25 and 587. We will use a Ubuntu server for the examples.

  • DNS records for our domain. How DNS records are manipulated depends on the DNS hosting provider (or DNS server configuration).

We’ll assume

Step 1: DNS Setup

The first step in setting up our e-mail forwarding server is to add MX, PTR and SPF DNS records for our server. This is really important as many e-mail providers, in order to prevent spam, will refuse to talk to mail servers without proper MX, PTR and SPF records set up. How to update DNS records will depend on the DNS hosting provider.

The following is what we need:

  • An MX record for example.com, pointing to myserver.example.com.

    This tells the world that e-mails to <whatever>@example.com should be delivered to the server myserver.example.com.

  • An A record for myserver.example.com, pointing to the IP address of our server.

  • A PTR (reverse DNS) record mapping the IP address of our server to myserver.example.com.

    This allows Gmail to verify the legitimacy of our server via its IP when Gmail receives a forwarded e-mail from it.

  • A TXT record, with key example.com and value v=spf1 mx ~all.

    This is an SPF record; it tells Gmail that the servers specified in the MX records of example.com, in this case only myserver.example.com, are allowed to send e-mails purporting to be from <whatever>@example.com. All other servers attempting to do the same will be rejected. This should be a sane default value, but feel free to custom it as you like.

DNS records will take a while (depending on our provider, up to a day) to propagate. Until they do, e-mails forwarded by our new e-mail server may get marked as spam or rejected outright.

Step 2: Receiving E-mail

We’ll first set up an e-mail forwarding server which will let we receive e-mails sent to our domains.

We will use Postfix as the e-mail server. Let’s start by installing the necessary packages. We will assume a Ubuntu server in the examples below; if you’re using a different distribution, please consult your distribution’s documentation for the right commands.

$ sudo DEBIAN_FRONTEND=noninteractive apt-get install postfix

In the above command, we skip the debconf configuration UI with DEBIAN_FRONTEND=noninteractive as we will edit the configuration files directly.

/etc/postfix/main.cf

Open up /etc/postfix/main.cf in your favorite editor. This file will come pre-populated with a bunch of config options and comments. Replace the contents of the file with the following (you can comment out the original contents for reference):

# /etc/postfix/main.cf

# Host and site name.
myhostname = myserver.example.com
mydomain = example.com
myorigin = example.com

# Virtual aliases.
virtual_alias_domains = example.com
virtual_alias_maps = hash:/etc/postfix/virtual

The first few lines are pretty straightforward; they tell Postfix how to identify itself to the world. The last two lines tell Postfix to forward e-mails sent to <whatever>@example.com to another e-mail provider (Gmail), and that the forwarding is configured in the database file /etc/postfix/virtual.

/etc/postfix/virtual

Let’s now open up /etc/postfix/virtual and fill in our forwarding configuration:

# /etc/postfix/virtual

# Forwarding mapping, one from-to address pair per line. The format is:
#     <forward-from-addr> <whitespace> <forward-to-addr>
[email protected]        [email protected]

We can add as many forwarding rules as you want, one on each line. We can use any number of tabs / whitespaces between the forward-from and forward-to addresses.

Update lookup table

It turns out that Postfix doesn’t actually read /etc/postfix/virtual (surprise!); instead, what it reads is a lookup table generated from it. So, let’s generate the lookup table from our /etc/postfix/virtual:

$ sudo postmap /etc/postfix/virtual

Note: we must re-run this command every time we modify /etc/postfix/virtual!

(Re)start Postfix

It’s now time to (re)start Postfix with our new configuration:

$ sudo postfix start
$ sudo postfix reload

Testing

We should test our brand-new Postfix server by sending an e-mail to our forward-from address. You might want to check out these tutorials for testing directly with the telnet command.

If you received the e-mail in your Gmail inbox, congratulations! Otherwise, check the Postfix logs at /var/log/mail.log (or with journalctl -u postfix if using Systemd) for errors. The most likely cause of issues is that DNS records have not propagated yet, in which case we’re likely to see a "rate limited" or "rejected for spam" type error message.

If we only want to receive but don’t care about sending e-mails as [email protected], then this is all we need.

Otherwise, let’s move on to configuring Postfix to support sending e-mails from Gmail.

Step 3: Sending E-mail

Gmail requires a relay server (a server that will send e-mails to their destination on behalf it) to speak TLS, which protects the communication between Gmail and the relay server.

We will use Cyrus SASL for this task. To install:

$ sudo apt-get install sasl2-bin libsasl2-modules

User name & password

Open relays are a terrible idea. We want to protect our e-mail server with a user name and password so that it will let Gmail send e-mails through it, but block spammers and other evil actors.

Cyrus SASL supports several backends for storing user names and passwords, including MySQL and PAM. However, we will go with the simplest backend — a plain database file.

Let’s create the user name / password database file in the default location /etc/sasldb2, with a single user named smtp (we can change the user name to anything we want):

$ sudo saslpasswd2 -c -u example.com smtp

Verify with:

$ sudo sasldblistusers2

Now, we make sure only Postfix can read this file:

$ sudo chmod 400 /etc/sasldb2
$ sudo chown postfix /etc/sasldb2

Lastly, we tell Cyrus SASL to use the file-based database to authenticate. Create the file /etc/postfix/sasl/smtpd.conf:

pwcheck_method: auxprop
auxprop_plugin: sasldb
mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5 NTLM
log_level: 7

SSL Certificate

We need an SSL certificate to enable TLS. While a proper SSL certificate signed by a Certificate Authority (CA) can be pricey, it turns out that a simple self-signed certificate suffices and works just fine.

So, let’s generate one.

  1. Generate an RSA private/public key pair. Note that you MUST supply a password to this command; the password will be removed in step 3.

    $ openssl genrsa -des3 -out example.key 1024
    
  2. Generate a Certificate Signing Request (CSR). Make sure to enter myserver.example.com when prompted for the "Common Name".

    $ openssl req -new -key example.key -out example.csr
    
  3. Remove RSA private/public key password.

    $ mv example.key example.key.orig
    $ openssl rsa -in example.key.orig -out example.key
    
  4. Generate a self-signed certificate. In the example, the generated certificate will be valid for 10 years.

    $ openssl x509 -req \
        -days 3650 -in example.csr -signkey example.key -out example.crt
    
  5. Create a PEM file.

    $ cat example.crt example.key > example.pem
    
  6. Move and protect the PEM file.

    $ sudo mv example.pem /etc/postfix/example.pem
    $ sudo chmod 400 /etc/postfix/example.pem
    $ sudo chown postfix /etc/postfix/example.pem
    

Note: Make sure to protect the generated private key and certificate with care!

Relay server

The final step is to configure Postfix to enable relaying of e-mail on behalf of Gmail.

Let’s open up /etc/postfix/master.cf. This file should already contain a bunch of config options, some of which are commented out. Uncomment the lines starting with submission and edit them to match the following:

submission inet n       -       n       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=may
  -o smtpd_tls_cert_file=/etc/postfix/example.pem
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

A Postfix restart is due after all these changes:

$ sudo postfix reload

If all went well, you should see Postfix serving a relay server, protected by our user name and password in /etc/sasldb2, on port 587.

Step 4: Configure Gmail

Finally, let’s log in to Gmail and tell it about our brand new server. (Note that the new Inbox UI does not yet support all the settings available in the legacy interface at the time of writing, so these instructions apply to the legacy interface.)

Let’s go to Settings > Accounts and Import and click Add another email address you own.

In the dialog that pops up, fill in our target e-mail address and click Next Step.
In the next dialog, enter the address of our e-mail server (myserver.example.com), and the user name and password we set up using saslpasswd2 above. Make sure the user name is suffixed with our domain name (so [email protected] rather than just smtp). Check that the correct port (587) and the correct security protocol (TLS) are selected, then click Add Account.
If all goes well, we should see a dialog like the following:
And we should receive an e-mail, forwarded by our mail server set up in the first part, to our [email protected] address. Click on the link inside the e-mail and we’re done!

Now, in any e-mail compose / reply window, you should be able to select [email protected] in the drop-down list next to "From". Congrats!

If, on the other hand, you see an error like the following:

You should check the Postfix logs at /var/log/mail.log (or with journalctl -u postfix if using Systemd) for any errors.

[Optional] Step 5: Setup DKIM and SRS

DKIM, short for DomainKeys Identified Mail, is e-mail validation system for preventing e-mail spoofing. Google recommends setting up DKIM; if we don’t, the e-mail messages that we forward to Gmail have a higher probability of getting marked as spam.

SRS, short for Sender Rewriting Scheme, is a relatively new standard that e-mail forwarding servers are recommended to adopt, and it also helps reduce the likelihood of our forwarded e-mail messages getting marked as spam.

Thus, while technically optional, it’s a good idea to configure our Postfix server to support both these standards. See my follow-up tutorial on Setting Up DKIM And SRS In Postfix for a detailed step-by-step guide.

Conclusion

In this post, we have set up a Postfix e-mail server that accomplishes the following:

  • E-mails sent to [email protected] is received by Postfix on myserver.example.com port 25, and forwarded to [email protected];

  • Gmail will let you select [email protected] as a "from" address when composing / replying to an e-mail, and will relay such e-mails through Postfix on myserver.example.com port 587 using a configured user name / password combination.

I hope you found this guide useful, and if you have any thoughts / questions, you’re more than welcome to leave a comment below. Cheers!

The Most Popular Fonts On The Web: A Study

If you’ve ever worked on a web site, you already know that choosing the right fonts is one of the most important aesthetic decisions in the design of a site. But, like all aesthetic decisions, it is a highly subjective.

I decided to try to bring a little bit of objectivity into the equation by finding out, empirically, what fonts the most popular sites on the web are using today.

Methodology

I wrote a Python program that crawls the front page of the 100,000 most popular web sites according to Alexa’s top sites list. It parses the HTML using BeautifulSoup, and parses all in-line and linked CSS stylesheets using cssutils. It then looks for font and font-family rules in the CSS rules, and stores the normalized form of each font in those rules in order. The result is kept in a SQLite database for analysis.

This all sounds pretty straightforward, but in fact it took me two weeks to build a crawler that doesn’t choke on all the crazy crap people throw at browsers. I will write up some of the more interesting cases in a separate post.

The final crawl success rate was about 96%. The size of all HTML and stylesheets downloaded was about 30GB.

First, let’s take a look at the first-choice/most preferred fonts, i.e., the ones that appear first in font-family rules. These fonts most closely represent the intention of the web site designers without compatibility compromises.

For each font, we calculate the number of distinct web sites that have at least one CSS font or font-family rule that lists the font as the first choice. This means that if a site has two rules, one listing Arial and another listing Times, it will count towards both. Thus, the numbers add up to much higher than the total number of sites.

Without further ado, here’s the breakdown of the top fonts on the web:

(The chart is interactive - hover/click to see actual numbers.)

A couple of observations:

  • Sans-serif fonts dominate the web. The top 25 fonts list only includes 4 serif fonts, compared to 16 sans-serif fonts. The most popular serif-font, Georgia, ranks #4 on the list with about 20% of sites.

  • Monospace fonts get no love. Most sites don’t bother specifying a custom monospace font; the most common monospace font specification just uses the browser default (12%). The most popular monospace font, Monaco, is featured on 7.2% of sites. Both of these are quite high, in fact, considering that we only crawl the front page of these sites.

  • The most popular fonts of each family:

    • Sans-serif: Arial (#1, 62%)

    • Serif: Georgia (#4, 20%)

    • Monospace: Monaco (#11, 7.2%)

  • Old Microsoft fonts still reign over the web. The top non-Microsoft font, Helvetica Neue, ranks #6 with an impressive 18% of top web sites.

  • Font Awesome is awesome. About 4.6% of the top 100K sites already use it for universal icons.

  • The Chinese web is on the rise. 3 out of the top 25 fonts are Chinese fonts, compared to 21 Latin fonts (and 1 symbol font). No other scripts made their way into the list.

What If We Considered…

  • Fonts in all positions: (graph) not much difference.

  • The top 1,000 web sites only: (graph) even more Arial (#1, 74%).

Next, let’s look at first-choice fonts used in headings or titles.

I used a really crude metric to determine whether a CSS rule matches a heading: whether the CSS selector is for an H1H6 tag or contains the strings "heading" or "title". While this finds only a subset of actual headings, it is not a bad approximation as it matches rules on about 58% of the top 100,000 web sites.

Let’s start with the graph:

Observations:

  • Header fonts are more diverse. While Microsoft fonts still reign supreme, a number or more exotic fonts are also on the list. In terms of distribution, there’s a much longer tail. It may be a sign that designers pay a lot more attention to fonts used in headings.

  • Serif fonts are more popular in headings than elsewhere. While Arial still claims to top spot with 27.31% of sites, the top serif font, Georgia, rises to 2nd place with 9% of sites.

  • No monospace fonts in headings. Not surprising.

  • Among Chinese fonts, 宋体 is more commonly used for body text, while 雅黑 (or Yahei) is more commonly used for heading text.

Concluding Thoughts

Arial and friends are the most hated fonts ever. Quoting The Scourge of Arial:

Despite its pervasiveness, a professional designer would rarely - at least for the moment - specify Arial.

And yet, Arial is still the default font on the vast majority of sites on the web, followed closely by its friends. Why?

Some people actually have a reason to use them but most use it mindlessly - just because everyone else does. Often, no thought is given to design of the site, let alone typography.

This is pretty sad. Paraphrasing Stop Using Lame Fonts, a good font stack has the potential to really make a site design shine, and it’s a shame web designers aren’t exploiting this opportunity.

Caveats

There are a couple of caveats with this dataset.

First, I am only surveying the landing page of each site. For many sites, notably those whose main interface is hidden behind a login (Facebook, Evernote, etc.), we may not be finding the styles that matter most to users. However, I figured since it would be pretty poor design to build a landing page that is aesthetically inconsistent with the rest of the site, it is not very likely that the font selection on the landing page would be too different from the fonts used elsewhere on the site. Of course, without creating fake accounts on these sites, we have no way to verify.

The numbers are not weighted by prominence on the web pages. One could argue that the font of the main body text on a page should carry more weight than that of the tiny disclaimer text at the bottom which no one reads. It would be tricky to determine what the right weight function is though.

The numbers are not weighted by the prominence of the web sites. For example, we could make it so that Google’s use of Arial would get more weight than some random obscure site’s use of Arial, as Google’s choice impacts many more users and was probably the result of deliberation by a team of expert designers. Again, it’s not entirely obvious how the weights should be assigned.

I am only looking at CSS rules in <style> tags or linked stylesheets. That means I am ignoring style="…" attributes in tags, <font> elements, or dynamically assigned fonts (i.e., through JavaScript). I would be surprised if this turns out to be a big loss though.

Next Steps

I can think of quite a few other useful questions one might find the answer to from this dataset:

  • What are the most common font pairings?

  • What are the popular choices for heading fonts, given that I’ve chosen font X as my body text font?

  • What are the most common fonts on news sites/forums/productivity web apps/social media sites?

  • Is there any correlation between font choice and site popularity?

What do you think?

Please feel free to download the top 100 first-choice fonts as a CSV file.

ConnectBot on the Asus Eee Pad Slider (SL101)

I recently acquired an Eee Pad Slider, a truly groundbreaking Android tablet running Honeycomb 3.2. Its unique sliding keyboard allows it to transcend the conventional definition of tablets as content consumption devices while preserving to perfection the intimacy of touch interface interaction. To be honest, however, up until last week I had always seen tablets as these flashy toys whose only purpose in life was to satisfy the vanity of a few mindless and unfortunately privileged gadget fanboys. But I fell in love with the Eee Pad Slider at first sight. This was a device that perfectly bridged that gap between my laptop and my smartphone; this was a device that was as immediately accessible, friendly, and visually appealing as my smartphone for casual use, but as pragmatic and dependable as my Thinkpad T400 when I need to get down and serious and reply to that one email or fix that one bug in my code. It’s simply the most versatile mobile device between a phone and a full blown PC.

Before I am dismissed as a crazed and dangerous fanatic, however, I should admit that I fully understand the fact that the Slider is not the best tablet for everyone. The Slider is probably twice as thick as the iPad or the Samsung Galaxy Tab and about the same price as either one of the two. Its keys are much less comfortable than those on your typical ultraportable laptop. It is also brown, a color I would not place very high on my list of good colors for electronic devices. It is probably not the best consumer lifestyle gadget you can find on the market, nor the most suitable for mobile office computing. But it is a very innovative and intelligent cross of the two worlds, and that happened to be exactly that one device I could not find in the hugely diverse and volatile market of mobile devices of today.

Back, then, to the real deal. I am a computer geek and I cannot live very well without SSH. On Android the SSH client that I (and it looks like everyone else) have been using is ConnectBot. So among the first apps I installed on my brand new Eee Pad Slider was ConnectBot; finally I thought, there was a mobile keyboard that had real control, shift, and alt keys that I could use. I was very surprised, therefore, when I realized that none of them worked. Given that the Slider has only been released for a month, forum threads and other helpful resources are nonexistent. I dug up the discussion on this document and found Lorant Kurthy’s fork of ConnectBot on GitHub that fixed the same issues for the Eee Pad Transformer. I cloned his repository and, with a one-line change in the source code, enabled his fix for the Slider as well. So there we go - here is my version of ConnectBot with fixes for the Eee Pad Transformer and Eee Pad Slider hardware keyboards. As far as I can tell it works fine; in fact, I am typing this post on the Eee Pad Slider using Vim in an SSH session to my laptop via my version of ConnectBot. All the credit goes to Lorant Kurthy though.

QEMU ARM boot tags

I had to fight against a "feature" in QEMU today as I was screwing around. Normally when QEMU ARM boots a Linux kernel it sets up a tag list with all sorts of good information like available memory regions as explained here. This is the Linux’s standard for ARM and any ARM bootloader that loads Linux must do this. However, when I was booting my custom kernel with qemu-system-arm I discovered that R1 and R2 were empty upon boot using the switch -monitor stdio:

QEMU 0.13.0 monitor - type 'help' for more information
(qemu) info registers
R00=00000000 R01=00000000 R02=00000000 R03=00000000
R04=00000000 R05=00000000 R06=00000000 R07=00000000
R08=00000000 R09=00000000 R10=00000000 R11=00000000
R12=00000000 R13=00000000 R14=00000000 R15=00000020
PSR=400001d3 -Z-- A svc32

This meant of course that the kernel refused to load at all. In addition, nowhere in memory could the tag list be found, as I saw with a quick assembly search through memory space.

So I downloaded the QEMU source code and found the relevant code in hw/arm_boot.c. The code there is pretty straightforward, and it turns out that QEMU ARM will only install a tag list if it determines kernel to be Linux. This is fine in itself, but the way it figures out if a given kernel is Linux is really stupid - as a comment in the file reads:

/* Assume that raw images are linux kernels, and ELF images are not.  */

This is really silly as one just objcopy -O binary any kernel into a raw image anyway. So I just set the initial assignment to is_linux = 1; in arm_load_kernel(…), compiled the ARM version, and copied it into my /usr/bin. Sigh.