Posts Tagged ‘linux’

Massive dumps with MySQL

Tuesday, January 12th, 2010

hurr. *insert FLUSH TABLES joke here*

I have a 2.5GB sql dump to import to my MySQL server. MySQL doesn't like me giving it work to do, and the box it's running on only has 3GB of memory. So, I stumbled across bigdump, which is brilliant. It's a PHP script that splits massive SQL dumps into smaller statements, and runs them one at a time against the server. Always the way: 10 lines into duct-taping together something to do the job for you, you find that someone else has done it rather elegantly.1

In short, we extract the directory to a publicly http-accessible location, stick the sql dump there and tell it to go.

In long, installation is approximately as follows:

avi@jup-linux2:~$ cd www
avi@jup-linux2:~/www$ mkdir bigdump
avi@jup-linux2:~/www$ chmod 777 bigdump
avi@jup-linux2:~/www$ cd bigdump/
avi@jup-linux2:~/www$ wget -q http://www.ozerov.de/bigdump.zip
avi@jup-linux2:~/www$ unzip bigdump.zip
avi@jup-linux2:~/www/bigdump$ ls
bigdump.php  bigdump.zip

Where ~/www is my apache UserDir (i.e. when I visit http://localhost/~avi, i see the contents of ~/www). We need permissions to execute PHP scripts in this dir, too (which I have already). We also need to give everyone permissions to do everything - don't do this on the internet!2

Configuration involves editing bigdump.php with the hostname of our MySQL server, the name of the DB we want to manipulate and our credentials. The following is lines 40-45 of mine:

// Database configuration
 
$db_server   = 'localhost';
$db_name     = 'KBDB';
$db_username = 'kbox';
$db_password = 'imnottellingyou';

Finally, we need to give it a dump to process. For dumps of less than 2Mb3, we can upload through the web browser, else we need to upload or link our sql dump to the same directory as bigdump:

avi@jup-linux2:~/www/bigdump$ ln -s /home/avi/kbox/kbox_dbdata ./dump.sql

Now, we visit the php page through a web browser, and get a pretty interface:

BigDump lists all the files in its working directory, and for any that are SQL dumps provides a 'Start Import' link. To import one of them, click the link and wait.

  1. Yes, you Perl people, it's in PHP. But it's not written by me. So on balance turns out more elegant. []
  2. Those permissions aside - anyone can execute whatever SQL they like with your credentials through this page. Seriously, not on the internet! []
  3. Or whatever's smaller out of upload_max_filesize and post_max_size in your php.ini []

Generating Fluxbox menus for VNC (Vinagre) connections

Wednesday, December 16th, 2009

One of the lovely things about Fluxbox is the text-driven menu. One of the nice things about Vinagre (Gnome's VNC client) is the xml-based bookmarks file. Here's a handy script to create a Fluxbox submenu out of your Vinagre bookmarks:

 
#! /usr/bin/perl
 
use strict;
use warnings;
use XML::Simple;
my $HOME = $ENV{ HOME };
 
my $bookmarks_file = "$HOME/.local/share/vinagre/vinagre-bookmarks.xml";
my $menu_file = "$HOME/.fluxbox/vnc_menu";
 
my $xml = new XML::Simple (KeyAttr=>[]);
my $data = $xml->XMLin("$bookmarks_file");
 
open(MENU, ">$menu_file") || die "Error opening \$menu_file: $menu_file $0";
 
print MENU "[begin]\n";
 
foreach my $b(@{$data->{"item"}}){
	print MENU "[exec] ($b->{name}) {vinagre $b->{host}:$b->{port}}\n";
}
print MENU "[end]\n";
close MENU;
 

PHP error on fresh install of PHPWiki:
Non-static method _PearDbPassUser::_PearDbPassUser() cannot be called statically

Thursday, November 12th, 2009

I've just installed PHPWiki 1.3.14-4 from the debian repositories and out of the box I got the following message on trying to log in to it:

Fatal error: Non-static method _PearDbPassUser::_PearDbPassUser() cannot be called statically, assuming $this from incompatible context in /usr/share/phpwiki.bak.d/lib/WikiUserNew.php on line 1118

The problem appears to be that, as of PHP 5.something, you're not allowed to have a function with the same name as a class. Apparently it's been a failure in E_STRICT mode for a while.

Anyway, the solution is to rename _PearDbPassUser() to something else, and then replace all calls to it with this new name.

I've done this and, so far, everything appears to work.

The function is defined in /usr/share/lib/phpwiki/lib/WikiUser/PearDb.php:

 
jup-linux2:/usr/share$ diff phpwiki.bak.d/lib/WikiUser/PearDb.php phpwiki/lib/WikiUser/PearDb.php
18c18
< function _PearDbPassUser($UserName='',$prefs=false) {
---
>     function _PearDbPassUserFoo($UserName='',$prefs=false) {
 

and is called in /usr/share/WikiUserNew.php:

 
jup-linux2:/usr/share$ diff phpwiki.bak.d/lib/WikiUserNew.php phpwiki/lib/WikiUserNew.php
1118c1118
< _PearDbPassUser::_PearDbPassUser($this->_userid, $this->_prefs);
---
>                 _PearDbPassUser::_PearDbPassUserFoo($this->_userid, $this->_prefs);
1157c1157
< _PearDbPassUser::_PearDbPassUser($this->_userid, $prefs);
---
>                 _PearDbPassUser::_PearDbPassUserFoo($this->_userid, $prefs);
2120c2120
< function PearDbUserPreferences ($saved_prefs = false) {
---
>     function PearDbUserPreferencesFoo ($saved_prefs = false) {
 

Munin plugins are really easy to write

Wednesday, November 11th, 2009

Munin plugins basically need to output variable names and values, and a little bit of config. They're tremendously easy to write.

My plugin is mostly useless - it graphs the value returned by /dev/urandom, and the random constants from debian and dilbert. Current graph is here and the code is as follows:

#! /bin/bash
 
case $1 in
        config)
        cat < < EOF
graph_category amusement
graph_title Random numbers
graph_vlabel value
debian.label debian
dilbert.label dilbert
urandom.label /dev/urandom
graph_scale no
EOF
        exit 0
esac
 
urandom=$(cat /dev/urandom | tr -dc '0-9' | head -c2)
 
echo "urandom.value " $urandom
echo "debian.value 4"
echo "dilbert.value 9"
 

Munin's plugins live in /etc/munin/plugins/, most of which are symlinks to scripts in /usr/share/plugins/. On restart, munin-node rechecks the plugins directory and loads any new plugins.
For a plugin called foo, munin-node will run foo configure first to get the configuration of the graph (which is passed to munin-graph), and then foo. For information as to graph configuration, see here.
It takes about 15 mins of collection for it to start making a graph, and you'll get more data every 5mins thereafter.

The script itself is mostly self-explanatory, except for:

- The values and the labels are linked by what occurs before the dot. If you define foo.label in the config output, that is what will be used to label the number that comes after foo.value in the 'normal' output. The munin tutorial sort-of hints at this, but only uses one variable.

- Munin doesn't care what order the variables come out in, it uses the labels to determine who's who. Similarly, it doesn't seem particularly fussed as to which flavour of horizontal whitespace is used.

*buntu 9.10

Thursday, October 29th, 2009

It's out! Torrents here and here.

Some notes from configuring rTorrent

Wednesday, October 28th, 2009

In anticipation of the new *buntus tomorrow, I'm configuring one of my servers as a torrent node for it, and for reasons unknown I've settled on rtorrent which is in the repos. The documentation is a little lacking (but does tell you how to do things like download torrents. Read it), and the most popular HowTo is quite verbose.

~/.torrentrc is the config file for rtorrent. The one from the Debian repos puts an example in /usr/share/doc/rtorrent/examples/rtorrent.rc, and apparently so does *buntu. You probably want to edit this. Path variables can be absolute or relative, and are generally relative and inside ./ by default. I've two directories, ~/torrents/ for the torrents and ~/.rtorrent/ for rtorrent's working directories.
Some variables I found to be handy:

scgi_local="~/.rtorrent/socket/rpc.socket"

Defines the socket file for scgi communication, which you only really need if you want external stats from it. I did, but haven't yet got round to using them.

session = ~/.rtorrent/sessio

The session directory, which lets multiple instances of rtorrent remember where they left off. Cannot be shared between instances.

Scheduling
rtorrent supports command scheduling, the syntax for which is occasionally documented. The syntax is approximately:

schedule = a,b,c,d

Where
a: What you want to call this scheduled command1.
b: How many seconds after rtorrent starts you want to first execute the command.
c: The interval for subsequent executions.
d: the command you want to execute.
This comes in most handy for auotmatically starting torrents in a directory:

schedule = watch_directory,5,5,load_start=~/torrents/*.torrent

Exactly what that does is left as an excercise to the reader (the example .rtorrent.rc explains, too).

  1. I don't know where this comes in handy later on []

Removing user list on Ubuntu Karmic log-on screen

Friday, October 23rd, 2009

This doesn't work any more
Install XDM instead. ;)

Karmic ships with a new version of GDM (2.28) which is rewritten, and by default presents a list of usernames, in much the same way as XP does by default. Lots of people dislike this. It's also currently lacking a graphical config tool (it is in beta...).

To change it, run this:

 
# gdm gconftool-2 --set --type boolean /apps/gdm/simple-greeter/disable_user_list true
 

This, I feel, is non-ideal since it just replaces the users list with a 'log in' Window and button which is completely superfluous - if I'm at the logon screen, I probably do want to log onto the PC, and the most logical thing for it to do is to be already asking for my username, ideally with that text box in focus. The previous login screen was pretty much ideal, and I'm not sure what benefit the new one has.

Source: https://answers.launchpad.net/ubuntu/+source/gdm/+question/86506

Why I use Linux

Sunday, May 10th, 2009

This is one of those things that's always difficult to explain. The bits of Linux that I miss on other platforms are not things that you immediately see as being of particular importance. The best example of this is the freedom - why would I care that I'm free to do as I please with this software? I've no idea, if I'm perfectly honest. But what follows is a list of the things I miss the most when I find myself on something else (most often WinXP):

Customisation
With very little in the way of limitations, I can make it look and behave how I want it to. Most obviously, I can customise the taskbars and notification areas, add monitoring icons, change colours, configure keyboard shortcuts and the like. But I can also tear out bits of the system that I don't particularly want, optionally replacing them with options I prefer. Or just use daft keyboard mappings where every key acts as if it was three to the left or something.
The important bit is that my Linux PC works in the way I want it to. When I'm on a Windows box, I must work in the way it wants me to.

Two Clipboards
In xorg, the most prolific window server on Linux, you have two clipboards. One is the 'normal' one which you manually insert stuff into through <ctrl>+<c> or <ctrl>+<shift>+<del> and pasted with <ctrl>+<v> or <ctrl>+<shift>+<ins>. The other is automatically populated by highlighting text with the mouse cursor and pasted with the middle mouse click. It's a fantastically quick way of copying and pasting.

Multiple Workspaces
If you've used them, you're probably nodding in agreement, if you haven't, it's difficult to explain. The best way I can think of is to imagine a KVM switch, but where all the monitors you switch between are those on a multi-monitor display. If anyone's got any better ways of explaining it, please let me know so I can stick it here.

Always On Top
I know Windows Task Manager can do it, but that's about the only window I've never wanted to keep on top of the others.

Text Based
Linux is mostly text-based in its configuration, and is generally blessed with at least three sets of wonderfully powerful text processing tools (the shell, Perl and Python). This means that batch changes to configurations are really easy, as is exporting particular bits of configuration and importing them to somewhere else. Also, the fact that the shell is as powerful as the clicky interface means that pretty much anything you might want to do can be automated. Which brings me on to:

Automation
Windows in particular seems to have a bit of an aversion to automating things. I don't know if it's just the Windows environments I've found myself in, the fact that most tasks are substantially easier (if more time consuming) with a mouse than on the command line, or because the batch file setup is crap, but there seems to be something about a Windows environment that lends itself very well to the pointless replication of manual work.
Linux practically forces you to automate things. Typing commands into a shell, while easy and quick, isn't particularly fun. And it's so obviously easily automatable that, well, you might as well.

Repositories
If I want a gopher client, I open up my repository client, search for 'gopher client', read the descriptions of what it shows up, download what looks best and start using it.
This software has come from the same place as the operating system, so I can trust that it is not malware, that the description is accurate and that it will be compatible with my system. It will also be upgraded when I update the system.
It's easier, more secure, faster and more convenient than the Windows way of trawling the net and trying to work out on what basis to judge the trustworthiness of a particular application. Though if I'm that way inclined, the repository system doesn't stop me doing that.

Hardware Compatability
Maybe I'm just unlucky, but most times I've installed Windows, I've then spent a couple of hours visiting manufacturer's websites trying to find drivers for my hardware so I can get anything at all working. Sometimes on a fresh install I have a working network, but generally I've needed to find graphics, wifi, usb, audio and various chipset drivers at the very least, quite often ethernet ones, too. It's not helped by the fact that Windows is apparently completely unaware of what hardware's in the box until you've installed the correct driver for it, which makes working out which driver you need more a game of chance than a methodical process.

Modular and integrated
The unix philosophy is for software to be small, simple and have a single well-served purpose. This is generally adhered to in the free software world, mostly because it's a pretty good approach.
For example, I have aspell installed. It's a spellchecker and it checks spelling. It does so in my web browser, mail client, IM client and office suite. If i typed in anything else, it'd probably work there, too. This means I only ever need to add words to the dictionary once, and it'll be considered valid everywhere. It also means that the people who develop my web browser and mail client can concentrate on developing good web browsers and mail clients, and leave the spell checking up to the people who develop spell checkers.

The Community and its support
It's massive. Or, perhaps rather, they're massive. And useful. I'm on a number of mailing lists and forums dedicated to sharing tips and helping people with free software. If I'm having trouble with anything Linux based, there're three or four pools of several thousand enthusiasts who are not only likely to know or be able to find an answer, but will probably enjoy doing it. And they run across the spectra of difficulty, use cases and user stupidity.
I've spent a long time trying to find similar for Windows, but there appears to be a jump from free home user level support to pay-for business use support.

No Marketing Department
One fantastically huge advantage of free software is that there's no incentive to make it popular. In contrast to commercial software where the aim to sell as many licenses as possible, the only possible aim of free software is to produce the best software possible. Any large free software project has this aim, since there is no other way to be popular.
This is brilliant, for several reasons. Firstly, the software tends to turn out pretty good. Secondly, marketing teams don't get anywhere near it, so design and feature decisions tend to be founded in reality. There's no 3d accelerated solitaire (to my knowledge), but there's the above list of features. Thirdly, upgrades are non-compulsory. You'll never have version 1.4 stop working because 1.5 is out. You might well find that security updates are no longer released and so you're advised to upgrade, but if you *really* like 1.4 you can hire a developer to patch the security holes and develop it in the direction you want it developed.

Using Synergy to share a keyboard and mouse across PCs

Saturday, May 9th, 2009

Synergy is a really neat way of using multiple computers at the same time, like a more convenient KVM switch (you do need to be physically close to all of them).

It basically allows you to have a monitor for each PC on your desk, and one keyboard and mouse with which to monitor them. Switching between PCs involves just moving the mouse pointer onto the relevant screen. It uses 'screens' rather than monitors, so you don't need to let it know if you've got a complicated multi-monitor setup on a host (or even if you add or remove monitors from one), and it allows for having screens of different sizes and for, say, the bottom half of one screen to line up with the top half of the next.

It's a server/client model - you have one server box into which you plug the keyboard and mouse. The rest connect over the network to it (in cleartext, don't do this where you don't trust the network).

I have two hosts. My work laptop is running Windows XP and has an external monitor plugged in to it. My testing PC is running Debian testing and is a PC proper. Here's my desk:
desk
jup-linux2 is the left monitor (attached to the PC to its left, running Debian), jup-rmt07 is the laptop on the right, which is also attached to the Sony screen in the middle. Since I take the laptop home with me occasionally, jup-linux2 is configured as a server, jup-rmt07 as the client (less to unplug).

Synergy is in the debian repositories, so it's just an apt-get install synergy. This provides two binaries, /usr/bin/synergyc and /usr/bin/synergys, which are the client and the server respectively.
To install it in Windows, you'll want to grab it from their Sourceforge page

Configuring and starting the server
So, having installed, we can configure! The configuration file can be arbitrarily named, mine's cunningly called ~/.synergy.conf and appears verbatim at the bottom of this. Synergy's really rather configurable, but I've never found much need for more than the basics, so I've an incredibly simple setup.

First, we define some 'screens'. Here we can also configure optons for screens, especially as regards the transference of caps-lock and num-lock statuses. I've no special requests, so I just list the hosts whose screens I want Synergy to manage:

section: screens
	jup-rmt07:
	jup-linux2:
end

Second, we define the links. For each monitor, we state what is at any edge of it. This is used to decide where to put the mouse pointer on leaving the screen, so it needs to be done in both directions - the fact that jup-linux2 is to the left of jup-rmt07 does not imply to synergy that jup-rmt07 is to the right of jup-linux2:

section: links
jup-rmt07:
	left = jup-linux2
jup-linux2:
	right = jup-rmt07
end 

If I have two screens at different heights, I can tell synergy that the top 30% of jup-linux2 lines up with the bottom 40% of jup-rmt07, for example:

jup-linux2
	right(70-100) = jup-linux2(0-40)
jup-rmt07
	left(0-40) = jup-rmt07(70-100)

Again, you always always always need to define the screen in both directions. The file is parsed by synergy to see what to do on leaving that particular screen - when you're in jup-rmt07 and move towards the left of the screen, it's only going to do anything if there's a left defined for that screen, irrespective of how many rights point there.
If my screens are above and below each other, up and down are used. A screen can have as many other screens round it as you like, by assigning percentages of edges to different screens.

Configuring the server under Windows involves a different process to use the same principles. Essentially, you build the same text file as above, but in a clicky gui. It's a little odd, but quite simple.

configuring Synergy server under Windows

To start the synergy server, we now run

synergys --config ~/.synergy.conf

Replacing '~/.synergy.conf' with the path to wherever the config file is saved.

Configuring and starting the clients

So, we have a server. Now, clients.
On Windows, synergy is only one executable, so we start that, select the 'Use another computer's shared keyboard and mouse (client)' option, stick the server's hostname or IP in the box, and click 'start'.
If you have a *nix client, the command to connect to a server at jup-rmt07 is synergyc jup-rmt07. synergyc provides a few options for changing the behaviour.

Starting Synergy automatically
Finally, I want them to start automagically on boot/login. For the linux host, this is relatively easy, add the following to your crontab:

@reboot synergys --config ~/.synergy.conf

And it'll be started on boot.
To start it under Windows, click the 'AutoStart' button. This will let you configure it to start it on login or, if you have the requisite permissions, start on boot.