Occassionaly we’ve been seeing messages like:

spam acl condition: error reading from spamd socket: Connection reset by peer

in /var/log/exim4/paniclog

But lately the problem has become much more persistent. I found out, that there’s some arsehole spammer sending us 500K spams every ca. 70 minutes. From /var/log/mail.log:

Jul 17 08:52:32 mail spamd[477]: spamd: identified spam (7.9/5.0) for spamd:1004 in 288.5 seconds, 505669 bytes.

Now since spamassassin would take that long (nearly 5 minutes!!!) to find out whether that mail is a spam or not, exim would simply timeout it’s connection to spamassassin and the result was the message seen above.

What I did was to use sa-compile to compile the SA rules to “native code” and to enable “Rule2XSBody”, which on Debian lives in /etc/spamassassin/v320.pre.

This seems to make SA quite a lot faster and to use less CPU.

If this won’t help we could also restrict the maximum size of messages to scan.

Yours Tomáš Pospíšek

Since I found Michel de Boer’s explanation of many of the fundamental SIP concepts in his 2005 dutch presentation of the Twinkle SIP phone very short and easy to follow - contrary to a lot of other material about SIP out there - I’ve translated it to english.

Here it is: Twinkle - a SIP softphone for Linux

Well, sometimes one wonders why people write tools that wheight a megabyte, when it’s easy to stick together one or two tools with a script and have the same thing.

Here’s a ruby script that generates a ping latency graph (target is google):

#!/usr/bin/ruby
#
# based on
#   http://oss.oetiker.ch/rrdtool/prog/rrdruby.en.html example
# with help of the tutorial
#   http://oss.oetiker.ch/rrdtool/tut/rrdtutorial.en.html
# and the docu
#   http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html
# 
# placed in the public domain by Tomáš Pospíšek

require "RRD"

$rrd    = `mktemp /tmp/tmp.XXXXXXXX`.chomp
$grafik = `mktemp /tmp/tmp.XXXXXXXX`.chomp
$start  = 0

def now()
  Time.now.to_i
end

def create()
  $start=now()
  RRD.create(
    $rrd,
    "--start", "#{$start - 1}",
    "--step", "#{1}",       # every 1s
    "DS:pinglatency:GAUGE:10:0:200", # 10s of missing data are OK, min=0, max=200 (ping time)
    "RRA:AVERAGE:0.5:1:#{60*30}") # 60 seconds * 30 minutes
  puts "RRD database created #{$rrd} at #{$start}"
  puts "Your graph is at #{$grafik}"
end

def ping_latency
   `ping -c 1 -n -w 1 google.com` =~ /.* time=(\d+)/
  # 64 bytes from 74.125.43.147: icmp_seq=1 ttl=52 time=31.6 ms
  return $1
end

def update()
  puts "updating #{$rrd} with #{now()}:#{ping_latency()}"
  RRD.update($rrd, "#{now()}:#{ping_latency()}")
end

def generate_gfx()
  # puts "generating graph #{$grafik}"
  RRD.graph(
   "#{$grafik}",
    "--title", " Ping latency to Google",
    "--x-grid", "SECOND:10:MINUTE:1:MINUTE:2:0:%X",
    "--start", $start.to_s,
#    "--end", # -> now
    "--interlace",
    "--imgformat", "PNG",
    "--width=450",
    "DEF:pinglatency=#{$rrd}:pinglatency:AVERAGE",
    "AREA:pinglatency#00b6e4:ms")
end

#main
create()
while(true)
  update()
  generate_gfx()
  sleep 1
end

It generates a graph like this:

How do you test web applications? You can use Selenium, but Selenium is very heavy.

I tried wget.

To find out what exactly to send to the app, I started Firebug inside Firefox, went to the “Network” section and recorded what I would do, to see, whether the application works. Then I took the accessed URLs together with the parameters and first tried to send them manually from the command line. Observe, that Firebug lets you copy the exact POST/GET parameter as was sent by the browser, when you right click on the POST/GET command in the Firebug Network log. Once I managed to redo all the steps manuall from the commandline I transferred what I did into a script and edited it until it was nice and would run through:

$ history > test_my_web_app

test_my_web_app now looks as follows:

#!/bin/bash
#
# check availability of:
url="http://www.example.org/my-app" 

# exit immediately if something goes wrong -> test fails!
set -e

echo "Testing $url" 

# save an error log in case there's an error so that
# we can analyse it further
error_log=$( mktemp )
cookies=$(   mktemp )

echo "In case the tests fail, you'll find the error log under $error_log" 
# send everything that goes to STDERR to the log
exec 2>$error_log

echo "Getting a cookie..." 
wget --save-cookies $cookies "$url" -O /dev/null

# we could put username/password directly into the script but we're cautuous and don't
echo -n "please enter user for $url: ";     read    user
echo -n "please enter password for $url: "; read -s pass
echo

echo "Logging in..." 
# if we logged in successfully, then the webapp will send us an html page
# that contains the following string:
to_verify="You're logged in" 
# POST to the webapp:
wget --load-cookies $cookies \
     --post-data "login=Login&user_login=$user&user_password=$pass" \
     --save-cookies $cookies \
     --keep-session-cookies \
     "$url/login" \
     -O - \
| grep -q "$to_verify" 

# now we're logged in

echo "Query component DB..." 
# First search the DB (parameters need to be URI encoded...)
num_components_found=$(
  wget --load-cookies $cookies \
       --save-cookies $cookies \
       --post-data 'action=set_params&crit%5Bcolor%5D=red&crit%5Bmin_size%5D=0' \
       "$url/prepare-picture" \
       -O - \
  | grep 'Number of matching components:' \
  | sed 's/.*Number of matching components:<\/td><td>//; s/<.*//' )

echo "Make sure that the app found something..." 
[ $num_components_found -gt 0 ]

echo "Generate and get picture..." 
wget --load-cookies $cookies \
     --save-cookies $cookies \
     --post-data 'index=0' \
     "$url/get-pic" \
     -O /dev/null

echo "Hooray, $url successfully tested" 

rm $error_log
rm $cookies
<pre>

Surprisingly easy, isn't it?
      
    
    
    
  

Here’s a patch:

# diff -u multi_http_responsetime.orig multi_http_responsetime
--- multi_http_responsetime.orig        2010-02-01 10:13:22.000000000 +0100
+++ multi_http_responsetime     2010-02-01 10:30:23.000000000 +0100
@@ -16,6 +16,9 @@
 use Time::HiRes qw( time );
 use IO::Socket;

+$FAILED=0;
+$INFINITE=999999;
+
 # ----- config -----
 push(@url_array, "http://www.google.de");
 push(@url_array, "http://www.t-online.de");
@@ -34,7 +37,7 @@
                                        PeerPort => 80,
                                        Proto => 'tcp'
                                        );
-       return(0) unless ($sock);
+       return($FAILED) unless ($sock);
        print $sock "GET $baseurl HTTP/1.1\nHost: $vhost\nConnection: close\n\n";
        while (<$sock>) {
                $data .= $_;
@@ -63,11 +66,16 @@
                $baseurl =~ s@^\w+://.+?(/)@\1@;

                $tick1 = time();
-               geturl;
+               $retval = geturl;
                $tick2 = time();

-               $tspent = $tick2-$tick1;
-               $msecs = ($tspent * 1000);
+               if ($retval == $FAILED) {
+                       $msecs = $INFINITE;
+                       }
+               else {
+                       $tspent = $tick2-$tick1;
+                       $msecs = ($tspent * 1000);
+                       }

                printf "timespent$i.value %.3f\n", $msecs;
                }

Since upgrading to Karmic I had the problem under Ubuntu that SIP calls under Twinkle would start get all chopped up every few minutes.

Turns out it’s NetworkManager that scans the wireless spectrum every two minutes for networks even if you are allready connected to some AP.

While people seem to be fixing the problem in the proper way, by making scanning interfere less with an active connection, there’s a good soul that provides patched versions of NetworkManager that gives up scanning while there’s an active connection to an AP.

See https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/373680

In https://bugs.launchpad.net/bugs/509411 I had the problem, that locking the screen would just not work any more. It turned out that gnome-screensaver must be perpetualy runnung in the background in order to be able to lock the screen.

Also this would affect the computer not locking the screen when suspending.

It might be that Gnome/Ubuntu are user-friendly, however the bloat is starting to annoy me. Why do I need to be running 214 processes, when the only thing that is actually being used is firefox and a terminal? Why do I need a process (gnome-screensaver) that is using up CPU cycles and virtual memory, when it’s actually never doing anything useful?

Thus I purged gnome-screensaver (8MB) and installed i3lock.

There are three things missing:

1) Make the laptop lock the screen when suspending:

$ cat /etc/pm/sleep.d/01_screenlock
#!/bin/bash
#
# 01_screenlock: lock screen at suspend with i3lock
# based on http://bbs.archlinux.org/viewtopic.php?id=68158
#

USER=henry
DISPLAY=:0.0
IS_ACTIVE="$( pidof i3lock )"

lock_screen() {
  # skip if i3lock is allready running
  if [ ! "$IS_ACTIVE" ]; then
       su - $USER -c "/bin/sh -c 'DISPLAY=$DISPLAY /usr/bin/i3lock'" &
  fi
}

case $1 in
   hibernate)
       lock_screen
       ;;
   suspend)
       lock_screen
       ;;
   thaw)
       # not required.
       ;;
   resume)
       # not required.
       ;;
   *) exit $NA
       ;;
esac

2) Add it to the panel:

$ cat ~/.gnome2/panel2.d/default/launchers/i3lock.desktop
#!/usr/bin/env xdg-open

[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Terminal=false
Icon=/usr/share/icons/gnome/scalable/actions/gnome-lockscreen.svg
Name=Lock the Screen
Exec=i3lock

3) integrate it into package management, so that I don’t accidentaly remove packages on which the screen locking depends:

$ cat control
Section: misc
Priority: optional
Standards-Version: 3.6.2

Package: lock-screen
Maintainer: Tomas Pospisek 
Depends: i3lock, pm-utils, gnome-icon-theme
Files: 01_screenlock /etc/pm/sleep.d/01_screenlock
 i3lock.desktop /home/henry/.gnome2/panel2.d/default/launchers/i3lock.desktop
Description: lock screen on suspend/hybernate
 bla
$ equivs-build control
$ sudo dpkg -i lock-screen_1.0_all.deb

Of course, it’s a hack. However it works and is easy to adapt. Let me know at tpo at sourcepole.ch in case you get to make it more generic.

That’s it. Saved runtime bloat. Saved disk space (i3lock is 60K large).

Building heartbeat on Solaris is always a pain. This time it took 4 hours for heartbeat 2.1.3 on x86 Solaris 10.

Needed packages (from Sunfreeware):

autoconf-2.65-sol10-x86-local
automake-1.11-sol10-x86-local
bison-2.4.1-sol10-x86-local
coreutils-8.1-sol10-x86-local
db-4.2.52.NC-sol10-intel-local
flex-2.5.35-sol10-x86-local
gcc-3.3.2-sol10-intel-local
glib-2.20.4-sol10-x86-local
gmp-4.2.1-sol10-x86-local
libgcc-3.4.6-sol10-x86-local
libiconv-1.11-sol10-x86-local
libintl-3.4.0-sol10-x86-local
libnet-1.1.2.1-sol10-x86-local
libsigsegv-2.6-sol10-x86-local
libtool-2.2.6b-sol10-x86-local
m4-1.4.12-sol10-x86-local
make-3.81-sol10-x86-local
pkgconfig-0.23-sol10-x86-local

Create file /usr/local/bin/gnome-config:

#!/bin/sh
echo "-I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -L/usr/local/lib -lglib-2.0"

chmod +x /usr/local/bin/gnome-config

Fix gcc headers:

cd /usr/local/lib/gcc-lib/i386-pc-solaris2.10/3.3.2/install-tools/
./mkheaders

Fix hardcoded paths:

ln -s /bin/perl /usr/local/bin/perl
ln -s /usr/local/bin/true /usr/local/bin/python

Patch lib/clplumbing/cl_reboot.c according to this thread.

Build package:

export LD_LIBRARY_PATH=/usr/local/lib
./ConfigureMe package --disable-crm  --disable-lrm --disable-dopd

Install it:

cd pkg/packages/
pkgadd -d . LXHAhb

There's an extension for Firefox called Twinklefox. However it doesn't run under FF 3.5. I've bumped the version in install.rdf, but twinkle won't get called correctly none the less.

I browsed FF's settings and ... made a little wrapper script to call twinkle:

$ cat /usr/local/bin/twinkle_callto
#!/bin/sh
#
# firefox will call twinkle_callto like this:
#
#   twinkle_callto callto:+41812528006

number=`echo "$1" | sed 's/callto://'`
exec /usr/bin/twinkle --call "$number"

There you go. Very easy.

Go to Preferences->Applications and add this script to handle "callto" URLs. That's it. You will be able to click on a callto: URL and be handed over to Twinkle.

If you want to know in which order RedHat will execute init scripts, then you can list them like this:

grep '#\s*chkconfig:' /etc/init.d/* | sed 's/:#//; s/\/etc\/init.d\///;' | awk '{ printf "%2s %2s %7s %s\n", $4, $5, $3, $1 }' | sort

which will produce on my system:

01 99   12345 sysstat                                                                                                                                     
02 99   12345 lvm2-monitor                                                                                                                                
03 97     235 sysstat                                                                                                                                     
05 95     345 kudzu                                                                                                                                       
06 99   12345 cpuspeed                                                                                                                                    
08 92    2345 ip6tables                                                                                                                                   
08 92    2345 iptables                                                                                                                                    
 0 99    2345 microcode_ctl                                                                                                                               
10 90    2345 network                                                                                                                                     
11 88    2345 auditd                                                                                                                                      
11 89       - netplugd                                                                                                                                    
12 87    2345 restorecond                                                                                                                                 
12 88    2345 syslog
13  1     345 setroubleshoot
13 87    2345 irqbalance
13 87    2345 mcstrans
13 87     345 portmap
13 87       - multipathd
14 86     345 nfslock
15 85    2345 mdmonitor
15 85       - mdmpd
18 85     345 rpcidmapd
19 69       - rpcsvcgssd
19 85     345 rpcgssd
20 80       - kdump
22 85     345 messagebus
25 75     345 netfs
25 88    2345 pcscd
27 73       - ypbind
28 72     345 autofs
30 74       - nscd
35 65       - krb524
41 89       - rdisc
 4 99    2345 readahead_early
50 50       - netconsole
55 25    2345 sshd
56 50     345 xinetd
58 74       - ntpd
60 20       - nfs
64 36       - mysqld
80 30    2345 sendmail
85 15    2345 gpm
85 15       - httpd
90 10    2345 xfs
90 10       - psacct
90 60    2345 crond
91 35       - smb
91 35       - winbind
95 05    2345 anacron
95 05       - saslauthd
95  5     345 atd
95  5       - conman
96 99       5 readahead_later
97 03     345 rhnsd
97 03     345 yum-updatesd
98 02     345 avahi-daemon
98 02     345 haldaemon
98 02       - avahi-dnsconfd
99 01    2345 smartd
99 95      35 firstboot

Tomáš Pospíšek

Copying a DVD under Linux wasn’t a problem for a long time. I always used k3b. However lately I failed to copy a DVD.

k3b would say: "failed to retireve all CSS keys" or in german "es konnten nicht alle CSS Schlüssel geholt werden". k9copy would say: "libdvdread: Error cracking CSS key for /VIDEO_TS/VTS_03_1.VOB (0x0031ccb6)!!"

First I thought k3b didn’t have access to libdvdcss. However it turns out that aparently some DVD drives won’t return data from some sections of a disc if the region code of the drive is not set.

My drive is a “HL-DT-ST DVDRAM GSA-T20N”. After setting the region code with "regionset" I was able to copy the DVD. Thanks a lot to the population of the internet that allowed me to slowly progress to a solution…

In short:

$ sudo apt-get install regionset
$ regionset
[...]
Would you like to change the region setting of your drive? [y/n]:y
Enter the new region number for your drive [1..8]:2
New mask: 0xFFFFFFFD, correct? [y/n]:y
Region code set successfully!

Region codes can be found here

Tomáš Pospíšek

When I run the statically bound Mojotron executable, as downloadable from sourceforge under Ubuntu Interpid, then it doesn’t recognize the arrow/cursor keys. I.e. changing the direction of the shots is not possible.

This can be “taken care of” by changing the Mojotron configuration file:

$ diff -u ~/.mojotronrc.orig ~/.mojotronrc
--- .mojotronrc.orig    2009-04-13 12:43:57.000000000 +0200
+++ .mojotronrc 2009-04-13 12:44:43.000000000 +0200
@@ -8,11 +8,11 @@
    section Player1 {
        usekey = 62 (type=integer);
        movement = wasd (type=string);
-       aiming = cursorkeys (type=string);
+       aiming = ijkl (type=string);
    }

Tomáš Pospíšek

Some of the more or less officially recommended documents describing how to set up SMTP time spam scanning suggests calling (?) Spamassasin from Exim as user “nobody”.

However under Debian the User nobody intentionally has no home directory and thus, depending on how you’ve set up Exim4 and Spamassassin, Spamassassin will try to save its settings in its $HOME, resulting in the following error (visible in /var/log/mail.log):

Mar 2 09:55:59 mail spamd[4896]: config: cannot write to /nonexistent/.spamassassin/user_prefs: No such file or directory

A posting in a often referred to thread has a few suggestions for setup. The problem there however is that they are mutually exclusive in that you get either scanning at SMTP time or scanning for each user individually.

I do want individual users to be able fine tune Spamassassin for their needs but I also want the main bulk of spam rejected at the SMTP connection.

Thus, as suggested in the posting above I did create a special spamd user:

adduser --disabled-password spamd

which is not used for running spamd. but only used by Exim4’s exiscan patch to check for spam. Therefore I replaced all the following lines from the howto:

spam = nobody:true

by

spam = spamd:true

Seems to work.

Siderant:

The reason I dropped sendmail 15 years ago was that the configuration was way more complex than learning a new computer language. Exim4 has got to the exact same point by now: Debian’s variant of Exim4 has dozens of interdependent config files, partially created from templates and being rebuild by generators with half-standardized variables etc. It’s a pain. Maybe I should have upgraded to postfix from exim3 instead.

Evidently the Mail community needs a variation of Greenspun’s Tenth Rule Of Programming:

“Any sufficiently advanced MTA program contains an ad-hoc, informally-specified reimplementation of sendmail’s configuration system”

more...

more...