Avi :)

I've had this knocking about for a while in various forms. Following TheOpenSourcerer's post, I figured I'd get it in while he's getting the flack.

About a year ago, I remember there being some rejoicing at the prospect of Canonical open-sourcing Launchpad, their bug/issue/ticket tracking web application. I also remember being a mite confused by it. Canonical is the company behind Ubuntu Linux, the popular open source operating system. Surely they, of all people, had opened the source from the start? What does it say when the company most loudly and successfully pushing open source as an efficient means of software development to your average computer user, develops its in-house software behind closed doors? And, accepting that, why is opening the source means for rejoicing? It is surely the belated Right Thing To Do. If anything, the response should have been along the lines of "Why so long?"

More recently, I decided that a hodge-podge of scripts to keep my files in sync between PCs wasn't a good idea, not least because it didn't actually work, and since my home PC and my laptop were both Ubuntu, and Ubuntu One seemed easy enough to install, that'd do the trick. So I installed it and started using it. Then I decided to get my work PC in on the game. And find this message:

Requirements: Because we want to give everyone using Ubuntu One the very best experience, we require that you run Ubuntu 9.04 (Jaunty Jackalope) or higher.

Which is something I don't think I've come across before - a Free Software company producing software and inventing restrictions. Why shouldn't Ubuntu One work on my Debian desktop?
This incompatibility for the sake of it is something I remember from Windows, and it's not a good memory. I know it's possible to write a client for it - the client is at least open source - but the message that I am required to use Ubuntu to use it? What good does that do anyone?

Most recently came the news that on the netbook edition Canonical have decided to drop OpenOffice.org (which *is* undeniably bloated) and use Google docs in its place. Google Docs is completely proprietary. It's about as closed source as software can get, since you can't even study its behavior, only those interfaces you're permitted with it.
Why wasn't AbiWord used, with it's online service, for example? Or a pared down OpenOffice, perhaps? Canonical has shown in the past that it has the developer hours to make fantastic, awesome, changes to software. Why not do that now?

Ubuntu is the most popular desktop Linux distro. I'm sure there are ways of counting such that Fedora wins, but if something's packaged for Linux, it's available in a Ubuntu-pointed deb. And so it occupies a unique position for free software - it's an opportunity to be a fantastic demonstration of what is possible with free software. It is possible to make commercial progress without restricting user freedom, and it is possible to make a wonderfully usable operating system under these conditions.

Except Ubuntu's not demonstrating that. It's showing that using a billionaire benefactor and a bunch of closed source software we can turn a free operating system into a mostly-freeish wonderful one.

And I'd rather like Canonical to stop doing that, and get back to making free software look good.

As I posted yesterday, I have a massive MySQL dump to import. I tried BigDump, but one of the tables kept producing errors and so BigDump would exit. I don't need the whole db imported, so I wrote this to split it by table. It produces a new sql file for every table it finds, numbered sequentially so if you process them in alphabetical order it's the equivalent of the whole dump. USE statements get their own files in the same sequence.

#! /usr/bin/perl
 
use strict;
use warnings;
use 5.010;
 
my $dump_file = $ARGV[0];
&usage() if !$dump_file;
 
say "using ".$dump_file;
 
my ($line, $table,@query, $file_number,$file_name);
my $line_number = 1;
my $find_count = 0;
 
open(DUMP_IN, "< $dump_file");
        while(<DUMP_IN>){
                my $line = $_;
                if (/^USE\s.(\w+)./){
                        say "changing db: ".$1;
                        $file_name = &make_file_name("USE_$1", "$find_count");
                        &write_USE($file_name, $line);
                        $find_count++;
                }elsif (/^-- Table structure for table .(.+)./){
			## If the current line is the beginning of a table definition
			## and @query is defined, then @query must be full of the previous
			## table, so we want to process it now:
                        if (@query){
                        $file_name = &make_file_name("$table", "$find_count");
                                open(OUTPUT, ">$file_name");
                                        foreach(@query){
                                                print OUTPUT $_;
                                        }
                                close OUTPUT;
                                undef @query;
                        }
                        $table = $1;
                        $find_count++;
                }
                next unless $table;
                push @query, $line;
 
                $line_number++;
        }
close DUMP_IN;
say $line_number;
 
## Subroutines!
sub write_USE() {
        my($filename, $line) = @_[0,1];
        open (OUTPUT, ">$filename");
        print OUTPUT $line;
        close OUTPUT;
}
 
sub make_file_name() {
        my ($type, $number) = @_[0,1];
        $number = sprintf("%05d", $number);
        $file_name=$number."_".$type.".sql";
        return $file_name;
}
 
sub usage() {
        say "Error: missing arguments.";
	say "Usage:";
	say "$0 [MYSQL_DUMP]";
        exit 1;
}
 

A small downside is that this replaces my 2.5Gb file with about 1800 smaller ones. A scripted importer is to follow.

hurr. *insert FLUSH TABLES joke here*

I have a 2.5GB sql dump to import to my MySQL server. MySQL doesn't like me giving it work to do, and the box it's running on only has 3GB of memory. So, I stumbled across bigdump, which is brilliant. It's a PHP script that splits massive SQL dumps into smaller statements, and runs them one at a time against the server. Always the way: 10 lines into duct-taping together something to do the job for you, you find that someone else has done it rather elegantly.¹

In short, we extract the directory to a publicly http-accessible location, stick the sql dump there and tell it to go.

In long, installation is approximately as follows:

avi@jup-linux2:~$ cd www
avi@jup-linux2:~/www$ mkdir bigdump
avi@jup-linux2:~/www$ chmod 777 bigdump
avi@jup-linux2:~/www$ cd bigdump/
avi@jup-linux2:~/www$ wget -q http://www.ozerov.de/bigdump.zip
avi@jup-linux2:~/www$ unzip bigdump.zip
avi@jup-linux2:~/www/bigdump$ ls
bigdump.php  bigdump.zip

Where ~/www is my apache UserDir (i.e. when I visit http://localhost/~avi, i see the contents of ~/www). We need permissions to execute PHP scripts in this dir, too (which I have already). We also need to give everyone permissions to do everything - don't do this on the internet!²

Configuration involves editing bigdump.php with the hostname of our MySQL server, the name of the DB we want to manipulate and our credentials. The following is lines 40-45 of mine:

// Database configuration
 
$db_server   = 'localhost';
$db_name     = 'KBDB';
$db_username = 'kbox';
$db_password = 'imnottellingyou';

Finally, we need to give it a dump to process. For dumps of less than 2Mb³, we can upload through the web browser, else we need to upload or link our sql dump to the same directory as bigdump:

avi@jup-linux2:~/www/bigdump$ ln -s /home/avi/kbox/kbox_dbdata ./dump.sql

Now, we visit the php page through a web browser, and get a pretty interface:

BigDump lists all the files in its working directory, and for any that are SQL dumps provides a 'Start Import' link. To import one of them, click the link and wait.

1. Yes, you Perl people, it's in PHP. But it's not written by me. So on balance turns out more elegant. [↩]
2. Those permissions aside - anyone can execute whatever SQL they like with your credentials through this page. Seriously, not on the internet! [↩]
3. Or whatever's smaller out of upload_max_filesize and post_max_size in your php.ini [↩]

I've updated the theme, and applied my handy modifications that make it more grey. Here's the diff on the css file, if you're wondering what I did (and for next time when I forget). I despise CSS, so it's all nice and easy.

The changes are to give the <pre> tags a grey (#EEE) background colour, and to make the posts individual white boxes on grey, rather than an all-white page.

 
avi@avi:whiteasmilk_1.8$ diff style.css style.css.original
18,35d17
< /* Added to make the code blocks pretty. Stupid CSS implementations mean this will break
<    any form of validity in order to actually make it work. Stupid browser people.
<    I can't remember where I got this from, but if you reckon I might've found it on your site
<    let me know and if I believe you I'll stick a URL here
< */
< pre {
< 	border 0
< 	padding: 0.2em 0.5em;
< 	background-color:#EEE;
< 	white-space:pre-wrap;
< 	white-space: -moz-pre-wrap;  /* Mozilla, since 1999 */
< 	white-space: -pre-wrap;      /* Opera 4-6 */
< 	white-space: -o-pre-wrap;    /* Opera 7 */
< 	word-wrap: break-word;       /* Internet Explorer 5.5+ */
< }
<
<
<
40c22
<   body {background-color:#c9c9c9;}
---
>   body {background-color:white;}
323,326c305
< border-bottom:15px solid #c9c9c9;
<   padding-left:10px;
<   padding-right:10px;
<   background-color:#fff;
---
>   border-bottom:1px solid #999;

So the original question is answered - it *is* possible to ride from Land's End to Lowestoft overnight on the night of the solstice.

It's also a really stupid idea. And you can still give money to homeless people. I've stolen Joel's pictures and stuck them online here.

While in Penzance, we checked the weather forecasts for Sunday and Monday nights. Monday, our original plan, was forecast snowfall for the entire night all along our journey, while the Sunday night was only sleety and haily. We decided that Sunday night was the better option.

Before the ride, Joel and I had different, but quite clear, ideas of what would be the biggest difficulty - I was only concerned with keeping warm, and Joel was more concerned about the prospect of not being able to see where he was going. Joel had therefore augmented his headlight with a pair of foglights, each of equal brightness to the original headlight, and wired them in rather neatly. He's also made himself a pair of nifty seven-way adjustable brackets for them.
I, meanwhile, packed enough luggage for a several week stay - the total luggage capacity of my bike is roughly equal to two decent sized backpacking rucksacks, and I'd filled them with warm (and occasionally waterproof) stuff. I'd also added a screen to my bike, and brought along some emergency heated grips.
In Penzance, I noticed that Joel had a point, and scurried off to Halfords where I bought the first things that looked like they'd fit, and rather than fashion a nice and stable bracket for them, cable-tied them onto my indicators. I spent rather a long time connecting them up, repeatedly trying to work out why there was no power available while not checking whether I'd blown the sensibly included fuse. I think Joel meanwhile reasoned that I had enough excess warm clothing to solve all of Crisis' problems, so he'd just nick some of mine if it was necessary. Though he did bring along a tent.
Thanks in no small part to my ignorance of the fuse we didn't leave Penzance until it was already dark. We got to Land's End for about 5:30, dawdled for a bit, and then left for Exeter.

On the way up to Exeter the piles of grit along the edge of the road rather quickly turned into piles of snow which wasn't exactly what we'd been hoping for - I'd inferred from the weather forecast that we'd not see snow until at least after the first stop. We pulled in to Whiddon Down services for dinner in the Little Chef. There was snowfall on the petrol station forecourt as we filled up, and the Little Chef car park was effectively a sheet of ice. We parked on the petrol station forecourt, and skated off to interrupt the waiters' game of football and fashion some duct-tape-and-bin-bag overshoes.

Leaving Exeter, Joel pulled over to readjust his headphones and when we went to leave I found I couldn't start my engine. On hitting the start button, the starter motor turned, the lights went dim and then nothing happened, which is pretty certainly down to a flat battery. I whipped out a multimeter to Joel's apparent surprise, and found that the battery was indeed flat. Fortunately, I've had this bike for some time, so there is a lot of wiring all over it that isn't doing much any more, and from this and a screwdriver we fashioned a set of jump leads. After some acrobatics getting the bikes close enough (I didn't have any long spare wiring), the bike came to life again. I tested the charging circuit and found it was at 13V at 5000rpm, so the battery should have been charging. We assumed I'd been running the engine too slowly, which is a long-standing habit of mine, and causes not enough power to be generated to charge the battery.

Stonehenge was dark and cold. We stopped at the side of the road where we suspected Stonehenge would be - there were VW campers parked up in an otherwise attraction-free layby - and Joel took it upon himself to get a photo of him and the stones, while I attempted to coax my feet out of anesthesia. At the time, doing this while standing in several inches of snow didn't seem as counter-productive as it does in hindsight.
The stones were closed, and they even had security guards wandering round keeping people out. Apparently if we waited until dawn they'd let us in, though we mused that we'd arrived at midnight, which must be a significant time to someone.
In the meantime, I checked the temperature on the thermometer on my handlebars, which was reading -5.4 as we pulled up, and to my delight found that it had settled on a decidedly more humane -3.5. In celebration Joel cracked out the coffee and biscuits and demanded we stop somewhere with walls and a roof as soon as possible. Apparently he rather likes the ability to feel his feet.

So, we cheated and found ourselves on the M3 at about 1am. Winchester services were amazing. Firstly, I discovered that my charging circuit and battery were apparently fine - they just couldn't cope with my extra lighting - so I resolved to stop using the extra lights about 20 minutes before any stop to give the battery a chance to charge to the point where it'd be able to start the engine. Secondly, it had heating. I didn't realise how cold I was until I sat down and shook for a few minutes. Joel rudely disturbed the woman in the Costa shirt by asking her if she wouldn't mind making us a pair of coffees, but the guy in WHSmith was quite amused by the idea of company, and repeatedly told us it was a stupid idea.
I did some calculations as to how far on or off schedule we were (I asked the satnav), and found that if we went non-stop up the motorway, through London, and up the A12, we'd get there at abut 6:50am, with dawn happening from about 7:30 to 8:30. With this, and the fact it was jolly cold outside so we'd want to stop, in mind, we set off up the M3.

London was weird, it was the first real traffic we'd seen in about 500 miles and two days, and while feeling like home, we were still about 150 miles from the end. Though by the time we got to Trafalgar square, I found I'd fallen into the mindset that we'd pretty much finished.
We got into London at about 3:30am, and were at Trafalgar Square at 4, as London was waking up in its very businesslike manner, surrounded by festivities. We left London knowing we had just to go up the A12 which be both knew as a far as Ipswich, and then some, and so we should be at Lowestoft well within three hours.

So we left London in good spirits. The first 50 miles or so weren't too bad; we were riding through misty rain on unlit roads with a whole lot of traffic coming in the other direction with maladjusted headlights. In short, I could generally see to just ahead of my headlight, sometimes even as far the vehicle in front. It was about here that I realised how lucky we'd been with the traffic and the weather so far.
Since it was a Sunday night and not much would be open, we'd agreed to pull into every open petrol station, on the grounds that several quick stops would be preferable to none at all. The first stop was pretty icy, but usable. I stocked up on those little energy shots (don't get the Lucozade one, it's vile), and we cleaned visors and headlights.
From there, it pretty much went downhill. Every petrol station was iced over and/or closed until Ipswich, at which point I'd decided the massive Tesco Extra would be pressed into service as a stop. That, too, was icy, so we pressed on.

And then we ran out of grit. From Ipswich, no-one had deigned to grit the roads. We were on the dual-carriageway A12 and suddenly the overtaking lane disappeared under several inches of snow, and shortly afterwards a line of snow appeared down the middle of the remaining lane. This was apparently fine for the car drivers who were behind us, but on a bike you can't really afford to just slip a bit on one wheel, so we were down to about 30mph on a national speed limit dual carriageway. I decided we'd pull over the next time it was sensible, and do whatever we could to make this more humane.
About 5 miles later, the dual carriageway became a single carriageway, which meant that the steel central reservation was replaced with a snow one and the track we had through the snow was replaced with a strip of shallower slush about the width of a car down the middle of the road. By now, I couldn't feel my feet or the footpegs, and instead of nudging the gear changer with my toes I was stomping on it with my heel, with only a vague idea of where it was relative to my feet. I was keeping an eye on Joel in my wing mirrors, since we'd agreed that he'd flash his lights if he wanted to stop and wasn't in a position to overtake and pull over, but from when the countdown on the satnav ticked over to 20 miles to Lowestoft, I think I stopped watching most things, and was overcome with quite some determination to just get to the finish, and go somewhere warm and dry.
That 20 miles wasn't really indescribably bad, but it's difficult to make it sound realistic. So, as a substitute: it was ruddy painful, cold and petrifying and I've never been so happy to see a wind turbine as I was when we arrived at Ness Point. There never was a point to pull over, sensible or otherwise.

We got to Ness Point as dawn was breaking, where our dad was to meet us with a car and a trailer to offer us a lift home. We pulled up and he wasn't there, apparently he'd seen the 400m or so of sheet ice leading up to it and thought we wouldn't be so stupid as to have ridden along it. I'd not given it a second thought.
I rang him, and he arrived and took us to have breakfast at the Asda cafe. At 7:30 am in Lowestoft, there didn't appear to be much open, and we weren't in the mood for looking. After amusingly managing to give us each exactly not what we'd ordered, we went back to the bikes to load the trailer.

After some to-ing and fro-ing working out which bike was to go where and how (Joel's tyres were antisocially wide), we got the bikes on and strapped down, just as the left tyre of the trailer went flat. Fortunately, there were a couple of guys from the local council in a van, apparently with some time to kill, who offered us a jack to help get the wheel off. They then took dad off to the local Kwik Fit to get the tyre changed while Joel and I strapped the bikes down and tried to avoid noticing the salt they were covered in.
Dad got back with a wheel, fitted it, and we had a delightfully warm, eventless, warm, comfortable, warm, dry and warm drive back through several hours of traffic. I was pretty happy to have taken him up on the offer of the trailer. And warm.

One of the lovely things about Fluxbox is the text-driven menu. One of the nice things about Vinagre (Gnome's VNC client) is the xml-based bookmarks file. Here's a handy script to create a Fluxbox submenu out of your Vinagre bookmarks:

 
#! /usr/bin/perl
 
use strict;
use warnings;
use XML::Simple;
my $HOME = $ENV{ HOME };
 
my $bookmarks_file = "$HOME/.local/share/vinagre/vinagre-bookmarks.xml";
my $menu_file = "$HOME/.fluxbox/vnc_menu";
 
my $xml = new XML::Simple (KeyAttr=>[]);
my $data = $xml->XMLin("$bookmarks_file");
 
open(MENU, ">$menu_file") || die "Error opening \$menu_file: $menu_file $0";
 
print MENU "[begin]\n";
 
foreach my $b(@{$data->{"item"}}){
	print MENU "[exec] ($b->{name}) {vinagre $b->{host}:$b->{port}}\n";
}
print MENU "[end]\n";
close MENU;
 

I've had a silly idea, and figured I should get sponsorship for it: ride motorbikes from the most westerly point of the UK to the most easterly, during the night of the winter solstice.

Basically, we're to leave Land's End at about dusk on the 21st December and arrive at Lowestoft by dawn on the 22nd. Google reckons we need eleven and a half hours, I'm told we have twelve hours of darkness.

We'll be going via Stonehenge (it's designed for this time of year) and Trafalgar Square (whoo! big tree!). here's our proposed route.

I hate navigating the Dell website. It's inconsistent and messy and noisy, and all I generally want is a single date (when the warranty expires or expired on a given box). So I wrote this. It scrapes the Dell website, and returns the warranty info for the service tag it's been passed.
I've CGI'd it here.

#! /usr/bin/perl
 
use strict;
use warnings;
 
die "$0\n\tGet warranty info from dell.\nUsage\n$0 [SERVICE TAG]\n" if !$ARGV[0];
 
my $service_tag = $ARGV[0];
 
use LWP::Simple;
use HTML::TableExtract; # Is in the CPAN, and exists in the debian repositories as libhtml-tableextract-perl
 
## Make a URL:
my $url_base = "http://support.euro.dell.com/support/topics/topic.aspx/emea/shared/support/my_systems_info/en/details";
my $url_params = "?c=uk&cs=ukbsdt1&l=en&s=gen";
my $url = $url_base.$url_params."&servicetag=".$service_tag;
my $content = get($url);
 
# Tell HTML::TableExtract to pick out the table(s) whose class is 'contract_table':
my $table = HTML::TableExtract->new( attribs => { class => "contract_table" } );
$table->parse($content);
 
## Gimme infos!
foreach my $ts ($table->tables) {
	foreach my $row ($ts->rows) {
		print "", join("\t", @$row), "\n";
	}
}

Oddly, it's taken me until this afternoon to have real need for using getopts in Perl. After a not-overly-brief look around, I've settled on Getopt::Long for the purpose. It's marginally more complicated than the alternative (Getopt::Std), but more flexible and better at error checking.

To use it, you pass a hash of valid options to GetOptions, where the keys are options and the values are the references to variables in which to put their arguments.
The name of the option dictates what value(s) it can hold: the final character indicates type (i - integer, f - float, s - string), and the penultimate whether it is optional or not (= - required, : - optional). Flags are indicated by not following this pattern - they're just given a name with no symbols.

Getopt::Long allows for the shortest unambiguous switch to be used, doesn't distinguish between -o and --o, and allows for the negation of flags (if -flag sets a flag to 1, -noflag will set it to 0). It also doesn't touch @ARGV when it's done getting its flags out of it.

Here's a brief script hopefully helping explain the above:

#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long;
 
# This is only neccesary when using strict. Which is always.
my ($flag, $compulsory_string, $optional_string, $compulsory_integer, $optional_integer, $compulsory_
 
GetOptions(
        "o"=>\$flag,
        "cfloat=f"=>\$compulsory_float,
        "cint=i"=>\$compulsory_integer,
        "cstring=s"=>\$compulsory_string,
        "ofloat:f"=>\$optional_float,
        "oint:i"=>\$optional_integer,
        "ostring:s"=>\$optional_string,
);
 
print "flag set\n" if $flag;
print $compulsory_float."\n" if $compulsory_float;
print $compulsory_integer."\n" if $compulsory_integer;
print $compulsory_string."\n" if $compulsory_string;
print $optional_float."\n" if $optional_float;
print $optional_integer."\n" if $optional_integer;
print $optional_string."\n" if $optional_string;

I don't know if it's because I'm doing more with it than I used to, or the rose-tinted specs that come with the novelty value have worn off, or if the later updates have been designed for more powerful hardware, but my G1's been lagging a bit recently, so I figured I'd have a look at what I can do to make it faster. Turns out it's quite receptive to the idea of a swap file.
You'll need root access for this - everything below is to be run as root. Following is how I did it on my cyanogen'd G1, but I don't see why it wouldn't work on anything else - it makes use of standard linux tools. This will likely advance the death of your sdcard - it increases the reads and writes by some large margin. I don't know enough about sdcards to know how much quicker it will die, but modern ones will likely deal with it better than older ones.

Anyway, in your terminal emulator:

dd if=/dev/zero of=/sdcard/swap bs=1M count=100

You can call it whatever you like. The 'count' value (100 above) is how many MB of swap space you want. The above will create 100MB of swap.
This will take a few seconds (of the order of 50), so if it's 'hung' it's probably not crashed.

Then we make it into a swap file and activate it as swap space:

mkswap /sdcard/swap
swapon /sdcard/swap

Finally, we want to change the swappiness of the system. When the system memory is completely full, and more is requested, the kernel needs to work out how to create some space. It, generally, can do one of two things: drop some filesystem cache, or move some application memory to swap.
The swappiness value dictates the priority of either - a value of 0 means it will always try to drop fs cache rather than use swap, and a value of 100 means it will always try to use swap and preserve the fs cache. Only values between 0 and 100 inclusive are allowed, and neither guarantees any action - you can still find yourself swapping out at 0, for example - just the priority afforded each solution.
The default value for the Linux kernel is 60, which is generally not appropriate for something running completely from flash media (where the fs cache isn't so important). So I'm dropping it to 20, which seems to work for me. If it doesn't for you, try some other values, it's an on-the-fly change.

echo 20 > /proc/sys/vm/swappiness

On rebooting the phone, the above settings wont hold. The swap file will still be available (assuming the SD card is still inserted), but on reboot you'll need to run

swapon /sdcard/swap
echo 20 > /proc/sys/vm/swappiness

On the plus side, you'll be rebooting less.