12.09.09
Posted in How To, Intermediate, Mac at 10:47 am by Techie
On Linux and Windows, I use an extension called It’s All
Text to add an “edit” button to text areas. Hitting this will
launch the contents of the text area in my editor of choice — emacs.
(You can pick any editor you like.)
However, on Mac, it’s not really straightforward, due to the way they
handle applications. The comments in this
post pointed me in the right direction for how to set this up.
The default proportional font in aquamacs bugs me, so I fix that by
launching it with a textarea and going to Options -> Appearance
-> Font For Text-Mode and setting it to Andale Mono. I also turn
off Soft Word Wrap and Auto Word Wrap In Text Modes under options,
because I’m used to emacs’s old unfriendly behavior.
Permalink
05.21.08
Posted in How To, Intermediate, Linux at 3:52 pm by Techie
If you’re like me, you may need to build your own packages in Debian, or perhaps have downloaded different versions of packages to install on your system. Sometimes you’d like to go the other way, though, and install an older version of a package you found in the repository, or even the same version, but a different build.
For example, let’s say you built your own version of perl, say 5.8.8-7blindedbytech1. Then, Debian finally gets the same version of perl into their repositories, say 5.8.8-7etch3. Unfortunately, doing an apt-get dist-upgrade isn’t going to install this version. This is particularly annoying if other packages have dependencies on these perl versions.
What’s one to do? Wouldn’t it be nice if you could specify which version of perl package to install? As it turns out, you can. When you do the apt-get install of the package, follow it by an equal sign (=) and the particular package version you want, e.g:
sudo apt-get install perl=5.8.8-7etch3
Obviously, this will only work if the specific version is in one of your repositories. You can do this with multiple packages, too. In fact, I had to, because perl-base was also a different version:
sudo apt-get install perl=5.8.8-7etch3 perl-base=5.8.8-7etch3
Be careful of dependencies. Don’t just blindly run these commands unless you know what you’re doing, and be sure to read the output messages, as it may have unintended effects. For instance, for me, the above command uninstalled mysql-server. Make sure you read the warnings it gives you, and you understand what they mean.
Permalink
10.20.07
Posted in Advanced, How To, Linux at 4:35 pm by Techie
Compiz Fusion on a Radeon Mobility X1400 graphics card? Hell yeah!
Oh, driver hell. Proprietary drivers are the bane of Linux’s existence, as any Linux user with a current ATI graphics card can tell you. I have a Lenovo Z61m on which I’d like to install Ubuntu Gutsy Gibbon, but good luck — the ATI Radeon Mobility X1400 will wreak havoc on your installation procedure. This is not new; Feisty Fawn had the same problem. Fortunately, you can in fact work around it.
Installing From the Live CD
First, there’s getting the live CD to work. You can download the alternate install CD and work from text mode, but I’d much rather use the regular live CD. Fortunately, you can! Boot up the live CD, and it’ll eventually die at some point, because the X server can’t launch properly. Hit Cntrl + Alt + F2 to jump to a virtual console. Type clear to clear the screen, then type the following:
sudo apt-get update
sudo apt-get install xorg-driver-fglrx
sudo depmod -a
sudo aticonfig --initial
sudo aticonfig --overlay-type=Xv
startx
You’ll obviously need a working Internet connection to make this work. Congratulations, you can now use the live CD! Run the installer as per normal.
Special note for Lenovo users: Do not delete the FAT32 service partition! You can use this to restore your Windows partition or run other diagnostics. I found it worked well to resize the existing Windows partition and dual boot.
After installation, you’ll reboot your system and — surprise! It doesn’t work. It’ll die at loading the X server again. Hit Cntrl + Alt + F2, and enter the following:
sudo apt-get update
sudo apt-get install xorg-driver-fglrx
sudo apt-get install xgl
sudo apt-get install xserver-xorg-video-radeonhd
sudo depmod -a
sudo aticonfig --initial
sudo aticonfig --overlay-type=Xv
startx
This is slightly different than before. Why? Well, no doubt you want to use those fancy Compiz Fusion effects, right? You didn’t need them to do the installation, but installation is done now. You’ll need to get the right drivers, as well as install xgl. Once you’ve done that, you might also need to edit /etc/X11/xorg.conf to change Option "Composite" "0" to Option "Composite" "1". After that, you should be able to enable Visual Effects through System->Preferences->Appearance, Visual Effects tab.
Permalink
07.11.07
Posted in Error Messages, How To, Intermediate, Windows at 4:06 pm by Techie
I have a Lenovo X41 Tablet PC that I had installed Ubuntu onto. I recently decided to re-install the original Windows software, so I ordered the CDs from Lenovo and broke out an external CD drive. However, when I put in the Rescue and Recovery CD and booted the machine, I ended up with a blank black screen and nothing else. What gives? I updated the BIOS and the embedded controller firmwares, but to no avail. The computer would boot up the CD, a brief message about examining the setup would fly by, then a blank screen. I figured this was a problem with the recovery CDs, because my Ubuntu live CD booted right up.
As it turns out, Lenovo recovery CDs are very picky about their hard drive setup. They need to copy files to disk for a variety of tasks, so if you’re using up all the partitions, then the CDs will stop booting, but give you no other errors.
I had previously installed Linux on the machine, so my partition tables were in a state the machine didn’t really understand. What I needed to do was trash the existing partitions, and flush out the boot record.
Easy enough — I fired up the Ubuntu live CD, and launched the partition editor in System -> Administration. I had to turn off the swap space, by right-clicking on it in the partition editor, and selecting “swapoff”. By default, Ubuntu live CDs will scan your drives for partitions marked as swap space, and use it, so you have to turn off this usage to delete the partitions. I ran through and deleted all the partitions, and applied the changes.
I next had to clear out the master boot record, but I could do that right from the live CD. After doing that, I rebooted with the Lenovo Rescue and Recovery CD and said a little prayer.
Once I had done these steps, the recovery CD started right up. I’m surprised Lenovo doesn’t give more details on this in their support knowledge base.
Permalink
07.10.07
Posted in How To, Intermediate, Linux at 9:35 pm by Techie
In the old days, a techie might install Windows on a computer, then install Linux. Later, the techie might decide to trash the Linux partitions for some reason, but on reboot — oops! LILO was still there. These days, GRUB has taken the place of LILO, but the old problem is still there: how do you clear out the master boot record completely?
If you had a Windows 98 boot disk lying around, you probably remember the old magic of fdisk /mbr. This simple command did just what you wanted — flashed out the master boot record, and things worked like you expected.
However, in this day and age of Linux live CDs, there must be a way to do so without breaking out the floppies. As it turns out, there is, through a simple program known as dd.
The beginning of a hard drive is where you keep both the master boot record, as well as the partition table. What you want to do is write zeroes to the master boot record, but not the partition table. Many folks recommend writing zeroes to the first 512 bytes of the drive — this is wrong.
The truth is, the first 446 bytes of the drive are dedicated to the master boot record executable. The remaining 66 bytes are used to hold the partition table. Hence, what we need to do is read in a source of zeroes, and output it to the first 446 bytes of the drive. Simple! Let’s take a look at the command:
dd if=/dev/zero of=/dev/hda bs=446 count=1
This says to do just that — use as an input file the special device /dev/zero, an output file of the drive (substitute the drive identifier in question here!), with a block size of 446 bytes. Oh, and do it just once. No nonsense, quick, and effective. So, put away those MS DOS disks — that Linux live CD works just fine.
Permalink
06.11.07
Posted in How To, Intermediate, Linux, Mac, Windows at 11:24 am by Techie
What’s your favorite editor? Oh, I’m sorry, the correct answer should be emacs. Actually, you should use whatever editor you feel most comfortable with; I’ve just found the ability to record keyboard macros in emacs to be indispensable, all the more so since I’ve been using it for almost ten years. Be I writing on Linux, Windows, or Mac OS X, my preferred text editor is still emacs.
However, once I get on a new machine, I need to customize .emacs, which is the startup configuration file. There are a series of changes that I find quite useful, and often find myself looking up again and again. Without further ado, here are the customizations:
;; Emacs usually has a slash screen on startup. Let's get rid of that and start with a blank buffer.
(setq inhibit-startup-message t)
;; I like to see what is selected in the buffer. This turns on visual feedback on selections.
(setq transient-mark-mode t)
;; Activate font-lock mode (syntax coloring).
(global-font-lock-mode t)
;; Line numbers are good. Getting column numbering as well is better.
(column-number-mode t)
;; Scrollbars go on the right. Who puts these on the left?
(set-scroll-bar-mode 'right)
;; Enable mouse wheel scrolling.
(if (load "mwheel" t)
(mwheel-install))
;; Always end files in anewline.
(setq require-final-newline 't)
;; ...or ask to end in newline if needed
; (setq require-final-newline 'query)
;; Match parentheses. Useful to be sure you've closed everything up.
(show-paren-mode t)
; Display settings
; default size and color options for all frames.
(setq default-frame-alist
'(
; frame width and height
(width . 80)
(height . 40)
;;;; foreground, background, and cursor colors
;;; (foreground-color . "grey8")
;;; (background-color . "grey92")
;;; (cursor-color . "red3")
)
)
; places top left corner of initial frame at location (25,25) on screen
(setq initial-frame-alist
'(
(top . 25)
(left . 25)
)
)
; Temporary files cluttering up the space are annoying. Here's how we
; can deal with them -- create a directory in your home directory, and
; save to there instead! No more random ~ files.
(defvar user-temporary-file-directory
"~/.emacs-autosaves/")
(make-directory user-temporary-file-directory t)
(setq backup-by-copying t)
(setq backup-directory-alist
`(("." . ,user-temporary-file-directory)
(tramp-file-name-regexp nil)))
(setq auto-save-list-file-prefix
(concat user-temporary-file-directory ".auto-saves-"))
(setq auto-save-file-name-transforms
`((".*" ,user-temporary-file-directory t)))
I additionally have some macros that aren’t generally useful. There’s a million things to tweak in the customization, and if you’re fancy, you can even write your own hacks — better brush up on that lisp! Here are a few more resources of interest:
Permalink
02.09.07
Posted in How To, Intermediate, Linux at 2:33 pm by Techie
Do you have an ssh login to a server? How would you like to access that on your Linux desktop as if it were local? As it turns out, this isn’t hard to do, thanks to the power of FUSE (Filesystem in Userspace). Here’s the quick and dirty way to set this up on Fedora Core 6.
Install FUSE
First, you need FUSE, as well as the sshfs module. To do so, open up a command prompt, and do the following:
su -
yum install fuse
yum install fuse-sshfs
usermod -G fuse youraccount
Replace youraccount in the above with your user account. The last line adds you to the group fuse, although if you are in other groups, it’ll remove you from them. You’d want to specify all additional groups you’d want to be in, e.g. usermod -G fuse group1 group2 youraccount. If you’re logged in under your account, then you should log out and back in, so the group change takes effect.
Mount a Folder
Let’s say you want to create a mount point on your desktop so you can browse it. Do the following:
cd ~/Desktop
mkdir remote_folder
sshfs yourserver.someplace.not:/home/yourusername remote_folder
Replace yourserver.someplace.not with the ssh server you want to mount. Replace /home/yourusername with the remote folder you want to mount. If your local username differs from your remote username, you’ll have to specify it, e.g. sshfs remoteusername@yourserver.someplace.not:/home/yourremoteusername remote_folder
Once you do this, it should be all set — you can browse it from the desktop, and browse it from the command line, as if it were local.
Unmount a Folder
Unmounting is pretty simple; use fusermount -u mountpoint. For instance, in the above, you mounted an ssh share to ~/Desktop/remote_folder. To unmount this, just issue:
fusermount -u ~/Desktop/remote_folder
That’s all there is to it!
Permalink
11.10.06
Posted in How To, Intermediate, Linux at 7:22 pm by Techie
I decided to take the plunge and start playing around with dual core technology, so I went down to the local store and picked up a new CPU, RAM, and motherboard. However, what I discovered was that it can actually be quite difficult actually installing Linux to what I had purchased, due to the newness of the technology. While support for dual core CPUs has been around for a while, support for some of the more recent motherboards has not, and mine was one of them. So, I could wait for broader support in a few months, or see what I could do now.
First, the reference information — the hardware I used.
* Intel Core 2 Duo Processor E6600
* Intel Classic Series Desktop Board DG965SS
* Kingston KVR533D2N4K2/1G (2x 512 MB)
* Western Digital 74 GB SATA Raptor WD740ADFD
* Antec TruePower 2.0 TPII-430
* JVC XJ-HD166S ATAPI DVD-ROM
Live isn’t so live, after all.
My plan was to battle-test this system with a few different Linux distributions, and then transition to testing Windows, but alas — my plans were not to be. I couldn’t even successfully boot up the live CDs I had! I’ve reproduced the error messages below, for the people searching for an answer.
If I tried to boot up Knoppix 5.0 DVD (2006-06-01-EN), I would get this error:
Can't find KNOPPIX filesystem, sorry.
Dropping you to a (very limited) shell.
Press the reset button to quit.
If I tried to boot the Ubuntu 6.10 “Edgy Eft” live CD, I would get this error:
BusyBox v1.1.3 (Debian 1.1.1.3-2ubuntu3) Built-in shell (ash)
Enter 'help' for a list of built-in commands.
/bin/sh: can't access tty; job control turned off
If I tried to boot the CentOS 4.4 live CD, I wouldn’t even get an error message — it would just reboot when I tried to select an option.
The interesting thing about the above is that the file system for live CDs largely lives on the CD itself. Hence, if it couldn’t read it as in the above, then for some reason, the DVD drive wasn’t accessible. After doing a little research online, I discovered that the motherboard was relatively new, and did some rather funky things with the handling of the IDE devices. Hence, the computer would start to boot off the optical drive, but when it came time to load up a kernel and actually boot the system, the computer no longer knew how to access the drive! This all resulted from a lack of Linux support for the new motherboard architecture from Intel. Haven’t Linux developers built that time machine yet to get future architectures into old kernels? (Note that Windows would have similar problems; if I didn’t have an up to date Windows install CD or the driver disk, I would be out of luck.)
New hardware? Please stand by, support will be added momentarily…
Linux developers have a reasonably quick turnaround time on these things, and have, in fact, added support to the kernel. Unfortunately, they added it to the most recent kernel — 2.6.18. In CentOS 4.4 (as well as Red Hat Enterprise 4.4), this is 2.6.9-39. In Knoppix 5.0 is 2.6.17. In Ubuntu Edgy Eft, it’s 2.6.17. I would need to wait for the next release before getting support for the 965 chipset. Fortunately for me, Fedora Core 6 was released with the 2.6.18 kernel, meaning that I could use that instead. Being that I didn’t really feel like re-authoring my own custom live CD, I went with that, and will just have to wait for future releases on the others.
The devil is in the details.
I downloaded Fedora Core 6 for x86_64. I’m fairly certain that the i386 version would install in much the same manner, but decided that I might as well use the 64-bit version. I put in the DVD and booted up the computer.
However, it brought up a text-mode window to select language, keyboard, and installation method: something had gone wrong, because in a normal install, the full graphical environment would have been displayed. Upon selecting an installation method of “Local CDROM”, the installer reported the error “No driver found” — looks like I had some work to do.
First, I rebooted and struck F2 to get into the BIOS. Under the “Advanced” tab, “Drive Configuration”, I set “Configure SATA as” to “AHCI”. I then exited, saving changes.
Next, I rebooted with the the install DVD in the drive. When the message came up “To install or upgrade in graphical mode, press the <enter> key.”, I didn’t just hit enter — instead, I typed the following line:
linux all-generic-ide pci=nommconf
This is the magic trick. After skipping the media check, the full graphical environment came up at that point, and I could install as per normal. I did so, and the computer rebooted after installation.
So close, I can taste it!
The computer boots up, and the Internet connection was recognized straight away. The resolution was set to 800×600, so I went to System->Administration->Display, and clicked on the Hardware tab. I changed the monitor to “Generic CRT Display/Monitor 1280×1024″, and then I could set the resolution to (naturally) 1280×1024. System updates worked fine, and programs launched correctly. I could surf the web without issue (except for flash support, which is a whole other problem I won’t address here).
However, there was something wrong…the optical drive no longer worked! I could put in a CD or DVD, and they wouldn’t be recognized by the operating system, despite having worked fine for the install process. The problem here is in GRUB, the Linux bootloader. While the install had correctly added pci=nommconf to the loader, it had neglected to include the all-generic-ide tag.
I edited /etc/grub.conf as root, and changed the line…
kernel /vmlinuz-2.6.18-1.2798.fc6 ro root=/dev/VolGroup00/LogVol00 pci=nommconf rhgb quiet
…to…
kernel /vmlinuz-2.6.18-1.2798.fc6 ro root=/dev/VolGroup00/LogVol00 all-generic-ide pci=nommconf rhgb quiet
I needed to re-install GRUB, with grub-install /dev/sda1. A reboot later, and things worked fine.
Success!
So yes, Virginia, it is possible to install Linux on this wonky motherboard. What’s more, since the support is in the kernel (and far better people than myself are slavishly hacking away at it), it’s quite likely that distributions will support this out of the box in the future without special handling. I haven’t had any issues with this set up yet, and it runs in 64-bit mode to boot.
Some closing thoughts…
* The problems I experienced were largely a result of the IDE channel handling of the chipset. If I had an SATA optical drive, I could probably have gotten up and running on Fedora Core 6 without any extra futzing. This also may have gotten Ubuntu to install as well.
* This also means that if you have, say, ATA drives (not SATA), you may experience a whole host of other problems.
* Ubuntu “Edgy Eft” should have some backports built into it to handle the Intel 965 chipset, however, I couldn’t get them to work. It’s feasible that this CD could boot on an ATAPI optical drive, if given the right options.
* While these problems will probably go away in new releases, it’s not guaranteed. The chipset will at least be better supported.
* I’m not looking forward to the Windows installation. The chipset claims that I should be fine with Windows XP SP2 or Windows Media Center Edition. Only time will tell.
Permalink
09.14.06
Posted in Beginner, How To, Windows at 10:11 am by Techie
With the recent release of iTunes 7, lots of folks are upgrading and finding that their iPods are no longer recognized. This is actually a common problem, and not isolated to iTunes 7. Sometimes it just happens, even without upgrade. I found my iPod nano was no longer being recognized by iTunes after upgrade, but fortunately, the fix is pretty simple.
First, some details. After upgrading, I plugged in my iPod nano, and heard it detect the hardware. However, it never showed up in iTunes. I opened up My Computer, and found that the drive was being recognized, so Windows was acknowledging its presence. iTunes just no longer was recognizing its existence. I unplugged it and re-plugged it in, but it just wouldn’t pick up.
Apple recommends the five Rs when you’re having troubles: reset your iPod settings, retry on a different port, restart your computer (after updating software), re-install your iTunes software, and restore your iPod (erasing all data on it). Trying each of these steps (one at a time, not all of them) will solve most problems with iPods.
In my case, I first reset my iPod settings by scrolling to Settings->Reset All Settings (on the iPod itself, silly). Once I did this and my iPod rebooted, it was recognized by iTunes. Yay! However, the firmware was out of date, and iTunes requested I update to the latest. (Note: the old iPod updaters appear to no longer work — you now appear to have to use iTunes to do firmware updates.) Unfortunately, the update didn’t work. Here’s a tip: wait until your iPod finishes syncing. Once finished, I did a restore + update, to get it to the newest firmware. Once that was done, my iPod re-picked up, and I had to configure its sync settings anew, and leave it to copy music.
Permalink
08.31.06
Posted in Advanced, How To, Windows at 11:14 am by Techie
In the old days (say, around 1990) a must-have application when buying a computer was an encyclopedia on a CD-ROM. Hello, Grolier’s and Encarta! No more would you need a shelf full of books to look up interesting facts! When I bought an iBook, it came with a copy of World Book, which I thought quite an entertaining addition.
These days, such an addition is no longer the norm, thanks to the Internet. An incredible amount of information can be gleaned online with a quick search. However, a project started a few years ago has quickly risen to become a great resource for user-provided information on a wide variety of topics. I speak, of course, of Wikipedia. While initially just a quick repository for user feedback, it’s quickly become a resource worthy of comparison to more established sources, such as The Encyclopedia Britannica, even if its veracity may be in question.
I have a laptop, but don’t always have an Internet connection, but wondered, why can’t I have an offline copy of Wikipedia? As it turns out, I can. Now, if I’m on the road and want to look up something quickly, I don’t even have to find a hotspot — I can just turn on my laptop, pull up a browser, and find the answer. This article shows you how I did it.
Overview
Wikipedia runs on the open source software MediaWiki. This in turn runs on top of MySQL and PHP, as well as possibly Linux and Apache. My laptop runs Windows XP Professional SP2 Tablet PC Edition, so running Linux and Apache just wasn’t going to happen. Fortunately, there is a WAMP project (Windows – Apache – MySQL – PHP), which did all the hard work of that installation for me. So, all I’d have to do is:
- Install WAMP.
- Install MediaWiki.
- Download and install a pages dump of Wikipedia.
These instructions should in theory work for any Windows XP SP2 machine. However, your results may vary. I take no responsibility if you try this yourself! Some anticipated caveats:
- You need Administrator privileges. You’re installing software, as well as creating services, so you need the privileges.
- You need disk space. The full English Wikipedia will take a over 10 gigabytes when uncompressed into the database.
- You need NTFS. Because of this, the database files themselves may grow to larger than 2 GB. If you’re using FAT32, you’re out of luck.
- You’re installing a new service. By default, the server installs without remote access, and hopefully, you leave your firewall in place. However, you are still installing new services on your machine, which means they have the potential for exploitation.
- No pictures included. These instructions do not cover the images in Wikipedia.
That said, let’s get on with the show!
Install WAMP.
Go to the Wampserver site and download the latest WAMP distribution (in my case, 1.6.4). Double-click the executable to run, and the defaults will pretty much be what you want. (E.g., install to C:\wamp\, create a Start Menu group, do not auto-start, set DocumentRoot to www, and Launch immediately.)
A Windows Security Alert will probably pop up and ask if you want to keep blocking Apache HTTP Server. You want to select “Keep Blocking” for this question.
Now, in your systray on the lower right side you should see a little dashboard icon, with a lock on it. It should be white, and when you mouse over it, it should say “WAMP5 – All services running – server Offline”. (When they say “offline” here, they actually mean that it’s restricting access to localhost — it’s actually online, technically.
To verify that it’s working, open up a web browser, and point it at http://127.0.0.1/. If the installation was successful, you should see a page that looks like the following:
That’s it for WAMP!
Install MediaWiki.
First, we’ll set up a MySQL user for Wiki. To do so, make sure WAMP is running. (If not, go to Start->Programs->WampServer->Start Wampserver.) Then, go to phpMyAdmin. Click on “Privileges”, then “Add a new User”. Use the following values:
- User name:
wikiuser
- Host: Select “Local” from the dropdown
- Password: Select “Use text field” from the dropdown, and pick a password of your choice
- Generate Password: Click the “Generate” button
- Global privileges: Leave all unchecked
Scroll to the bottom and click “Go”, and it should successfully create a user. On this confirmation page, you should have a screen to edit the user if you scroll down. Do so, to the section marked “Database-specific privileges”. Set the dropdown to “Use text field”, and enter “wikidb”. Click “Go”.
You should be presented with a new page for Database-specific privileges. Click the “Check All” link to check all the boxes, and click “Go”.
Download the latest stable release of MediaWiki. At the time of this writing, that was version 1.7.1. It’s a .tar.gz file, so you’ll need a program to expand it — I recommend the shareware program WinRAR. When you unpack this, you’ll create a folder named mediawiki-1.7.1. Rename this to wikipedia, and move it to c:\wamp\www\.
If you now visit http://127.0.0.1/wikipedia/, you should get a splash page saying to “setup the wiki” first. Follow that link, and you should get a “Site config” page. I used these values for this form:
- Wiki name:
Wikipedia Offline
- (Admin) Password: custom password
- Database host:
localhost
- Database name:
wikidb
- DB username:
wikiuser
- DB password: same password used when creating MySQL user
The other defaults were fine. Once done, I went in Windows Explorer to C:\wamp\www\wikipedia\config\, and moved the file LocalSettings.php up one directory to C:\wamp\www\wikipedia\.
Another check of http://127.0.0.1/wikipedia/ should state that “MediaWiki has been successfully installed.”
Download and install a pages dump of Wikipedia.
You can download a copy of the English Wikipedia pages from http://download.wikimedia.org/enwiki/latest/. However, you should check this page, for the entry for “enwiki” first, to make sure the dump completed successfully. The file you will want is named enwiki-latest-pages-articles.xml.bz2. This contains all the article pages, but none of the revisions or history. You just want the articles, right? As of this writing, that file is around 1.5 GB, compressed.
If you don’t already, you should make sure you have Java installed. If you don’t, you can get it from http://java.sun.com/j2se/1.5.0/download.jsp. I usually just open a command window and type java and hit enter, and see if it just hangs. If it does, it’s probably installed, and I hit cntrl-C to cancel.
You’ll also need MWDumper. Download mwdumper.jar from http://download.wikimedia.org/tools/. Put this file and the wiki dump file in the same directory, say, c:\tmp\.
You’ll need to edit MySQL’s config file to increase the max_allowed_packet size. If you don’t, the import will most likely choke around the 49,000 article mark. This is quite annoying, because it kills the rest of the import. While you’re add it, you might as well change the innodb_log_file_size, which should modestly increase the import speed. To do so, go to c:\wamp\mysql\, right-click on my.ini, and select Open. This will open up the ini file in a text editor. Find the line innodb_log_file_size, and set this to 512M (was 10M in mine). Scroll to the bottom, and add the following line:
set-variable=max_allowed_packet=32M
Remember that little dashboard with the lock in your systray? Left-click on it, and a menu should pop up. Select MySQL->Stop Service. Wait a few seconds, then left-click on it again, and select MySQL->Start / Resume Service.
Before you import, you’re going to need to delete data in MySQL from the default installation. Otherwise, you’ll get errors about a dupe right at the start, and then none of the rows will import. Left-click on the dashboard with a lock in your systray, go to “MySQL”, and select “MySQL console”. You’ll be asked for a password, which by default is blank, so just hit enter. Enter in the following commands into the console:
use wikidb;
delete from page;
delete from revision;
delete from text;
quit
This will delete all pages in the wiki.
Open a command window by going to Start->Run, and typing in cmd. Type c: to change to the c: drive, and then cd c:\tmp\ to change to the directory where you put mwdumper.jar and the wiki dump file. You’re ready to do the import, but beware — this will take a long time. It’s best to start the process, then leave for a few hours. When you’re ready, type the following:
java -jar mwdumper.jar --format=sql:1.5 enwiki-latest-pages-articles.xml.bz2 | c:\wamp\mysql\bin\mysql -u wikiuser -p wikidb
This will begin the import process, and as noted, this will take a long time. There are over three million pages to process, so don’t expect it to finish right away. On a reasonably fast single processor machine (*not* my laptop), it took me over 24 hours.
Usage
Using Wikipedia Offline is pretty straightforward. If you haven’t already, start WAMP. (If you see the dashboard with a lock icon in your systray, and it’s white, then it’s running. If not, go to Start->Programs->WampServer->Start Wampserver.) Then, just fire up a web browser and browse to http://127.0.0.1/wikipedia/. If all goes well, it should be accessible just like Wikipedia, searches and all.
Anticipated Questions
- Why do this?
I’m not always connected to the Internet, and think Wikipedia is a great resource. Now I can take it wherever I want. I suppose if I were paranoid about Wikipedia tracking my searches, then I could do this and do all the searches I wanted offline. Doing it this way also seemed like a fun tech project.
- Is this legal?
Sure! Wikipedia offers all of their data for use by interested parties. All of the software involved is open source, except for Windows.
- Where are the pictures?
You can download a dump of the English Wikipedia images from here. Wikipedia doesn’t package these with the dump for two reasons: 1) the images might be copyrighted, so they don’t want to distribute them; and 2) the dump file would be huge. As of this writing, the dump file is about 75 GB, which was larger than the hard drive on my laptop.
- Isn’t it overkill to install full MediaWiki?
Yes, but it’s not nearly as much effort as you might think. Plus, with WAMP, I can experiment with other types of LAMP-based software. You can always build static pages if you’d prefer something a little more lightweight.
- Won’t the data fall out of date?
Yes, but I’m doing this more to just have a quick reference, rather than something that’s kept constantly up to date. In that sense, it’s similar to those encyclopedia CD-ROMs! Besides, the way the dumps are handled, you’re guaranteed to be slightly out of date. If you really need to be that current, you should probably be going online.
- How can updates be done?
I’m presuming I can just go in mysql and delete from the ‘page’, ‘revision’, and ‘text’ tables; download a new dump; and re-import using mwdumper.jar. I haven’t actually tried this, though.
- Can I use these instructions to run a wiki web site?
There are a few problems with this. First, the WAMP folk note that “WAMP5 is not meant to be a production server.” Also, running a web site takes a fair bit of security knowledge to prevent hacking, so you’ll get yourself in trouble if you just use it to publish on the ‘net. Finally, you can’t republish the contents of Wikipedia as your own site. So, technically, you could use the first few parts to set up wiki, but it’s not a good idea. You’re better off getting a proper web host that has a one-click install of wiki, such as DreamHost.
- How do I uninstall?
If you want to trash the whole thing, go to “Add or Remove Programs” in the Control Panel, and select WAMP5. Remove it, then be sure to delete C:\wamp\ as well.
Permalink
« Previous entries Next Page » Next Page »
