Avi :)

I've had this knocking about for a while in various forms. Following TheOpenSourcerer's post, I figured I'd get it in while he's getting the flack.

About a year ago, I remember there being some rejoicing at the prospect of Canonical open-sourcing Launchpad, their bug/issue/ticket tracking web application. I also remember being a mite confused by it. Canonical is the company behind Ubuntu Linux, the popular open source operating system. Surely they, of all people, had opened the source from the start? What does it say when the company most loudly and successfully pushing open source as an efficient means of software development to your average computer user, develops its in-house software behind closed doors? And, accepting that, why is opening the source means for rejoicing? It is surely the belated Right Thing To Do. If anything, the response should have been along the lines of "Why so long?"

More recently, I decided that a hodge-podge of scripts to keep my files in sync between PCs wasn't a good idea, not least because it didn't actually work, and since my home PC and my laptop were both Ubuntu, and Ubuntu One seemed easy enough to install, that'd do the trick. So I installed it and started using it. Then I decided to get my work PC in on the game. And find this message:

Requirements: Because we want to give everyone using Ubuntu One the very best experience, we require that you run Ubuntu 9.04 (Jaunty Jackalope) or higher.

Which is something I don't think I've come across before - a Free Software company producing software and inventing restrictions. Why shouldn't Ubuntu One work on my Debian desktop?
This incompatibility for the sake of it is something I remember from Windows, and it's not a good memory. I know it's possible to write a client for it - the client is at least open source - but the message that I am required to use Ubuntu to use it? What good does that do anyone?

Most recently came the news that on the netbook edition Canonical have decided to drop OpenOffice.org (which *is* undeniably bloated) and use Google docs in its place. Google Docs is completely proprietary. It's about as closed source as software can get, since you can't even study its behavior, only those interfaces you're permitted with it.
Why wasn't AbiWord used, with it's online service, for example? Or a pared down OpenOffice, perhaps? Canonical has shown in the past that it has the developer hours to make fantastic, awesome, changes to software. Why not do that now?

Ubuntu is the most popular desktop Linux distro. I'm sure there are ways of counting such that Fedora wins, but if something's packaged for Linux, it's available in a Ubuntu-pointed deb. And so it occupies a unique position for free software - it's an opportunity to be a fantastic demonstration of what is possible with free software. It is possible to make commercial progress without restricting user freedom, and it is possible to make a wonderfully usable operating system under these conditions.

Except Ubuntu's not demonstrating that. It's showing that using a billionaire benefactor and a bunch of closed source software we can turn a free operating system into a mostly-freeish wonderful one.

And I'd rather like Canonical to stop doing that, and get back to making free software look good.

As I posted yesterday, I have a massive MySQL dump to import. I tried BigDump, but one of the tables kept producing errors and so BigDump would exit. I don't need the whole db imported, so I wrote this to split it by table. It produces a new sql file for every table it finds, numbered sequentially so if you process them in alphabetical order it's the equivalent of the whole dump. USE statements get their own files in the same sequence.

#! /usr/bin/perl
 
use strict;
use warnings;
use 5.010;
 
my $dump_file = $ARGV[0];
&usage() if !$dump_file;
 
say "using ".$dump_file;
 
my ($line, $table,@query, $file_number,$file_name);
my $line_number = 1;
my $find_count = 0;
 
open(DUMP_IN, "< $dump_file");
        while(<DUMP_IN>){
                my $line = $_;
                if (/^USE\s.(\w+)./){
                        say "changing db: ".$1;
                        $file_name = &make_file_name("USE_$1", "$find_count");
                        &write_USE($file_name, $line);
                        $find_count++;
                }elsif (/^-- Table structure for table .(.+)./){
			## If the current line is the beginning of a table definition
			## and @query is defined, then @query must be full of the previous
			## table, so we want to process it now:
                        if (@query){
                        $file_name = &make_file_name("$table", "$find_count");
                                open(OUTPUT, ">$file_name");
                                        foreach(@query){
                                                print OUTPUT $_;
                                        }
                                close OUTPUT;
                                undef @query;
                        }
                        $table = $1;
                        $find_count++;
                }
                next unless $table;
                push @query, $line;
 
                $line_number++;
        }
close DUMP_IN;
say $line_number;
 
## Subroutines!
sub write_USE() {
        my($filename, $line) = @_[0,1];
        open (OUTPUT, ">$filename");
        print OUTPUT $line;
        close OUTPUT;
}
 
sub make_file_name() {
        my ($type, $number) = @_[0,1];
        $number = sprintf("%05d", $number);
        $file_name=$number."_".$type.".sql";
        return $file_name;
}
 
sub usage() {
        say "Error: missing arguments.";
	say "Usage:";
	say "$0 [MYSQL_DUMP]";
        exit 1;
}
 

A small downside is that this replaces my 2.5Gb file with about 1800 smaller ones. A scripted importer is to follow.

hurr. *insert FLUSH TABLES joke here*

I have a 2.5GB sql dump to import to my MySQL server. MySQL doesn't like me giving it work to do, and the box it's running on only has 3GB of memory. So, I stumbled across bigdump, which is brilliant. It's a PHP script that splits massive SQL dumps into smaller statements, and runs them one at a time against the server. Always the way: 10 lines into duct-taping together something to do the job for you, you find that someone else has done it rather elegantly.¹

In short, we extract the directory to a publicly http-accessible location, stick the sql dump there and tell it to go.

In long, installation is approximately as follows:

avi@jup-linux2:~$ cd www
avi@jup-linux2:~/www$ mkdir bigdump
avi@jup-linux2:~/www$ chmod 777 bigdump
avi@jup-linux2:~/www$ cd bigdump/
avi@jup-linux2:~/www$ wget -q http://www.ozerov.de/bigdump.zip
avi@jup-linux2:~/www$ unzip bigdump.zip
avi@jup-linux2:~/www/bigdump$ ls
bigdump.php  bigdump.zip

Where ~/www is my apache UserDir (i.e. when I visit http://localhost/~avi, i see the contents of ~/www). We need permissions to execute PHP scripts in this dir, too (which I have already). We also need to give everyone permissions to do everything - don't do this on the internet!²

Configuration involves editing bigdump.php with the hostname of our MySQL server, the name of the DB we want to manipulate and our credentials. The following is lines 40-45 of mine:

// Database configuration
 
$db_server   = 'localhost';
$db_name     = 'KBDB';
$db_username = 'kbox';
$db_password = 'imnottellingyou';

Finally, we need to give it a dump to process. For dumps of less than 2Mb³, we can upload through the web browser, else we need to upload or link our sql dump to the same directory as bigdump:

avi@jup-linux2:~/www/bigdump$ ln -s /home/avi/kbox/kbox_dbdata ./dump.sql

Now, we visit the php page through a web browser, and get a pretty interface:

BigDump lists all the files in its working directory, and for any that are SQL dumps provides a 'Start Import' link. To import one of them, click the link and wait.

1. Yes, you Perl people, it's in PHP. But it's not written by me. So on balance turns out more elegant. [↩]
2. Those permissions aside - anyone can execute whatever SQL they like with your credentials through this page. Seriously, not on the internet! [↩]
3. Or whatever's smaller out of upload_max_filesize and post_max_size in your php.ini [↩]

I've updated the theme, and applied my handy modifications that make it more grey. Here's the diff on the css file, if you're wondering what I did (and for next time when I forget). I despise CSS, so it's all nice and easy.

The changes are to give the <pre> tags a grey (#EEE) background colour, and to make the posts individual white boxes on grey, rather than an all-white page.

 
avi@avi:whiteasmilk_1.8$ diff style.css style.css.original
18,35d17
< /* Added to make the code blocks pretty. Stupid CSS implementations mean this will break
<    any form of validity in order to actually make it work. Stupid browser people.
<    I can't remember where I got this from, but if you reckon I might've found it on your site
<    let me know and if I believe you I'll stick a URL here
< */
< pre {
< 	border 0
< 	padding: 0.2em 0.5em;
< 	background-color:#EEE;
< 	white-space:pre-wrap;
< 	white-space: -moz-pre-wrap;  /* Mozilla, since 1999 */
< 	white-space: -pre-wrap;      /* Opera 4-6 */
< 	white-space: -o-pre-wrap;    /* Opera 7 */
< 	word-wrap: break-word;       /* Internet Explorer 5.5+ */
< }
<
<
<
40c22
<   body {background-color:#c9c9c9;}
---
>   body {background-color:white;}
323,326c305
< border-bottom:15px solid #c9c9c9;
<   padding-left:10px;
<   padding-right:10px;
<   background-color:#fff;
---
>   border-bottom:1px solid #999;

One of the lovely things about Fluxbox is the text-driven menu. One of the nice things about Vinagre (Gnome's VNC client) is the xml-based bookmarks file. Here's a handy script to create a Fluxbox submenu out of your Vinagre bookmarks:

 
#! /usr/bin/perl
 
use strict;
use warnings;
use XML::Simple;
my $HOME = $ENV{ HOME };
 
my $bookmarks_file = "$HOME/.local/share/vinagre/vinagre-bookmarks.xml";
my $menu_file = "$HOME/.fluxbox/vnc_menu";
 
my $xml = new XML::Simple (KeyAttr=>[]);
my $data = $xml->XMLin("$bookmarks_file");
 
open(MENU, ">$menu_file") || die "Error opening \$menu_file: $menu_file $0";
 
print MENU "[begin]\n";
 
foreach my $b(@{$data->{"item"}}){
	print MENU "[exec] ($b->{name}) {vinagre $b->{host}:$b->{port}}\n";
}
print MENU "[end]\n";
close MENU;
 

I hate navigating the Dell website. It's inconsistent and messy and noisy, and all I generally want is a single date (when the warranty expires or expired on a given box). So I wrote this. It scrapes the Dell website, and returns the warranty info for the service tag it's been passed.
I've CGI'd it here.

#! /usr/bin/perl
 
use strict;
use warnings;
 
die "$0\n\tGet warranty info from dell.\nUsage\n$0 [SERVICE TAG]\n" if !$ARGV[0];
 
my $service_tag = $ARGV[0];
 
use LWP::Simple;
use HTML::TableExtract; # Is in the CPAN, and exists in the debian repositories as libhtml-tableextract-perl
 
## Make a URL:
my $url_base = "http://support.euro.dell.com/support/topics/topic.aspx/emea/shared/support/my_systems_info/en/details";
my $url_params = "?c=uk&cs=ukbsdt1&l=en&s=gen";
my $url = $url_base.$url_params."&servicetag=".$service_tag;
my $content = get($url);
 
# Tell HTML::TableExtract to pick out the table(s) whose class is 'contract_table':
my $table = HTML::TableExtract->new( attribs => { class => "contract_table" } );
$table->parse($content);
 
## Gimme infos!
foreach my $ts ($table->tables) {
	foreach my $row ($ts->rows) {
		print "", join("\t", @$row), "\n";
	}
}

Oddly, it's taken me until this afternoon to have real need for using getopts in Perl. After a not-overly-brief look around, I've settled on Getopt::Long for the purpose. It's marginally more complicated than the alternative (Getopt::Std), but more flexible and better at error checking.

To use it, you pass a hash of valid options to GetOptions, where the keys are options and the values are the references to variables in which to put their arguments.
The name of the option dictates what value(s) it can hold: the final character indicates type (i - integer, f - float, s - string), and the penultimate whether it is optional or not (= - required, : - optional). Flags are indicated by not following this pattern - they're just given a name with no symbols.

Getopt::Long allows for the shortest unambiguous switch to be used, doesn't distinguish between -o and --o, and allows for the negation of flags (if -flag sets a flag to 1, -noflag will set it to 0). It also doesn't touch @ARGV when it's done getting its flags out of it.

Here's a brief script hopefully helping explain the above:

#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long;
 
# This is only neccesary when using strict. Which is always.
my ($flag, $compulsory_string, $optional_string, $compulsory_integer, $optional_integer, $compulsory_
 
GetOptions(
        "o"=>\$flag,
        "cfloat=f"=>\$compulsory_float,
        "cint=i"=>\$compulsory_integer,
        "cstring=s"=>\$compulsory_string,
        "ofloat:f"=>\$optional_float,
        "oint:i"=>\$optional_integer,
        "ostring:s"=>\$optional_string,
);
 
print "flag set\n" if $flag;
print $compulsory_float."\n" if $compulsory_float;
print $compulsory_integer."\n" if $compulsory_integer;
print $compulsory_string."\n" if $compulsory_string;
print $optional_float."\n" if $optional_float;
print $optional_integer."\n" if $optional_integer;
print $optional_string."\n" if $optional_string;

I don't know if it's because I'm doing more with it than I used to, or the rose-tinted specs that come with the novelty value have worn off, or if the later updates have been designed for more powerful hardware, but my G1's been lagging a bit recently, so I figured I'd have a look at what I can do to make it faster. Turns out it's quite receptive to the idea of a swap file.
You'll need root access for this - everything below is to be run as root. Following is how I did it on my cyanogen'd G1, but I don't see why it wouldn't work on anything else - it makes use of standard linux tools. This will likely advance the death of your sdcard - it increases the reads and writes by some large margin. I don't know enough about sdcards to know how much quicker it will die, but modern ones will likely deal with it better than older ones.

Anyway, in your terminal emulator:

dd if=/dev/zero of=/sdcard/swap bs=1M count=100

You can call it whatever you like. The 'count' value (100 above) is how many MB of swap space you want. The above will create 100MB of swap.
This will take a few seconds (of the order of 50), so if it's 'hung' it's probably not crashed.

Then we make it into a swap file and activate it as swap space:

mkswap /sdcard/swap
swapon /sdcard/swap

Finally, we want to change the swappiness of the system. When the system memory is completely full, and more is requested, the kernel needs to work out how to create some space. It, generally, can do one of two things: drop some filesystem cache, or move some application memory to swap.
The swappiness value dictates the priority of either - a value of 0 means it will always try to drop fs cache rather than use swap, and a value of 100 means it will always try to use swap and preserve the fs cache. Only values between 0 and 100 inclusive are allowed, and neither guarantees any action - you can still find yourself swapping out at 0, for example - just the priority afforded each solution.
The default value for the Linux kernel is 60, which is generally not appropriate for something running completely from flash media (where the fs cache isn't so important). So I'm dropping it to 20, which seems to work for me. If it doesn't for you, try some other values, it's an on-the-fly change.

echo 20 > /proc/sys/vm/swappiness

On rebooting the phone, the above settings wont hold. The swap file will still be available (assuming the SD card is still inserted), but on reboot you'll need to run

swapon /sdcard/swap
echo 20 > /proc/sys/vm/swappiness

On the plus side, you'll be rebooting less.

I've just upgraded to Cyanogen on my G1 and it's lovely. Well, I got root, which is basically what I always wanted.

I basically followed the instructions on the Cyanogen wiki and everything worked exactly as described, I've nothing really to add here except to say it's brilliantly easy and everyone should do it.
One small note, though, the final reboot into Cyanogen takes a long time. Mine took just shy of thirty minutes, and I've read of others taking anything above about 15.

The first obvious benefits are root access, exchange activesync support, five workspaces (as against the stock 3) and a bunch of useful apps already installed, including a nifty power control widget which replaces my collection of 5 distinct widgets. I'll write more on it when I've used it more.
In case, as I did, you've been pondering this and wondering what the end result is like, it's almost exactly as previously, but with some new features, there's some screenshots on the webpage. I've not yet found anything to have been degraded by it, and it all looks and feels the same, but a bit more polished in areas. Though it only ships with one ringtone.

I've just installed PHPWiki 1.3.14-4 from the debian repositories and out of the box I got the following message on trying to log in to it:

Fatal error: Non-static method _PearDbPassUser::_PearDbPassUser() cannot be called statically, assuming $this from incompatible context in /usr/share/phpwiki.bak.d/lib/WikiUserNew.php on line 1118

The problem appears to be that, as of PHP 5.something, you're not allowed to have a function with the same name as a class. Apparently it's been a failure in E_STRICT mode for a while.

Anyway, the solution is to rename _PearDbPassUser() to something else, and then replace all calls to it with this new name.

I've done this and, so far, everything appears to work.

The function is defined in /usr/share/lib/phpwiki/lib/WikiUser/PearDb.php:

 
jup-linux2:/usr/share$ diff phpwiki.bak.d/lib/WikiUser/PearDb.php phpwiki/lib/WikiUser/PearDb.php
18c18
< function _PearDbPassUser($UserName='',$prefs=false) {
---
>     function _PearDbPassUserFoo($UserName='',$prefs=false) {
 

and is called in /usr/share/WikiUserNew.php:

 
jup-linux2:/usr/share$ diff phpwiki.bak.d/lib/WikiUserNew.php phpwiki/lib/WikiUserNew.php
1118c1118
< _PearDbPassUser::_PearDbPassUser($this->_userid, $this->_prefs);
---
>                 _PearDbPassUser::_PearDbPassUserFoo($this->_userid, $this->_prefs);
1157c1157
< _PearDbPassUser::_PearDbPassUser($this->_userid, $prefs);
---
>                 _PearDbPassUser::_PearDbPassUserFoo($this->_userid, $prefs);
2120c2120
< function PearDbUserPreferences ($saved_prefs = false) {
---
>     function PearDbUserPreferencesFoo ($saved_prefs = false) {