Ok, I've got this up and running the way I want now, so I figure I'll document both for myself and for the greater good.

Goals:
* Run multiple mediawiki sites using a single codebase and sharing users and login sessions (log in once for all wikis)

Assumptions:
* A very basic understanding of the shell (commands like cd, mkdir, tar)
* You have at least a minimal understanding of mediawiki installation and can
  get one site up and running
* Apache 2 (though these methods may work with version 1, it's untested)
* Apache runs as group apache
* A single MySQL database for the wiki running on localhost.
* Root access to your webserver
* Latest version of mediawiki (1.13.2 at the time of this writing) tarballed in
  your home directory.
* Write access to /home/www (see Conventions) for your normal user account.
* You'll set up one wiki called "main" and create other wikis named whatever.
  If you don't want to have a "main" wiki, just change the naming of the
  first wiki to one of the wikis you would like to install.

Conventions:
* Paths will live under /home/www -- modify for your environment
* /home/www is NOT a DocumentRoot for apache (including any virtualhosts, etc)
* Commands will be inside quote tags and start with "$" for user commands and
  "#" for root commands
* Commands may be split over multiple lines for simplicity with a backslash (\).
* Text to go into files will also be inside quote tags, but will not be
  prefixed with $ or # (except for comments, it should be obvious which is
  which).
* Variables that you should change will be all caps and are self-documenting
  (e.g., YOUR_IP_ADDRESS)
* Database names, user names, etc can be altered to your liking. I'll just use
  what's conventional.

References:
* See the MediaWiki Installation Guide [mediawiki.org].

First, we'll set up two instances of mediawiki. One is kept pristine, and is just for running config/index.php to create all the database tables and LocalSettings.php. This wouldn't be totally necessary, but it makes for a nice way to do create new wikis without exposing yourself to nastiness during the config stage.

Give apache write access to the wikiadmin/config directory.

If you're paranoid (like me), you can remove the config files from the wikicode directory as we won't ever use them.

Now we have two installations of the code. One in the "wikicode" directory and one in the "wikiadmin" directory. For the sake of saving space, you can pare down the wikiadmin installation if you like. Personally, I removed everything under lanuages/messages except for MessagesEn.php. You can probably trim a lot more if you want.

It is absolutely critical that neither of these directories are configured as a DocumentRoot for your site, whether virtualhost or main configuration. Personally, I use an empty directory /home/www/wiki as my DocumentRoot for the wiki VirtualHost.

Next, edit your apache config to give you (and only you) access to your wikiadmin directory. If you are using a virtualhost, this goes inside the <VirtualHost> declaration, otherwise it can live in your main configuration:


Create your wiki database, and give the wiki access to it. You can do this however you like, but I prefer the simple CLI interface. Enter your mysql root user's password when prompted.

From here, you can set up your first wiki. Personally, I created a "Main Wiki." This first wiki setup will also defined the shared database tables, though any of them really could be used.

Go to http://YOURSITE_COM/wikiadmin [YOURSITE_COM]
Follow the configuration setup as normal. Most of the settings you can choose for yourself, I'll just mention a few important ones.

Wiki name: Main Wiki
Admin username: WikiSysOp
Password: YOUR_WIKI_ADMIN_PASSWORD
Password confirm: YOUR_WIKI_ADMIN_PASSWORD
Database type: MySQL
Database host: localhost
DB username: wikiuser
DB password: YOUR_WIKI_DATABASE_USER_PASSWORD
DB password confirm: YOUR_WIKI_DATABASE_USER_PASSWORD
Database table prefix: main_

And that's it. Make sure the rest of the settings are the way you want them, then click "Install MediaWiki!"

Next, move the LocalSettings.php you just created into the wikicode root directory. Be sure to rename it to LocalSettings-main.php! Also, be sure to move the file not just copy it. We don't want a LocalSettings.php hanging out in wikiadmin/config anymore.

Now you need to create the real LocalSettings.php. In your favorite text editor, just create /home/www/wikicode/LocalSettings.php to contain the following:

You can edit the html to present something nicer if you want. It really shouldn't happen that anyone sees that, unless you screw up your config.

Now you need to edit a few things in LocalSettings-main.php. These will be the things you change for each wiki you create. If these variables don't exist, add them.


If you want to enable uploads (and keep them seperate per wiki), then you'll first need to create an uploads directory and make it writeable by apache.

Then in LocalSettings-main.php change or create the variables:


Now we're all set. Update your apache config to serve the new wiki (in VirtualHost directive or main config).


Your first wiki is now ready to go. Visit http://YOURSITE_COM/main [YOURSITE_COM] to see it in action!

Now that we have all the difficult crap out of the way, adding new wikis is pretty simple. Let's say we're setting up a new wiki called "anotherwiki."

Go to the /wikiadmin site to create a new LocalSettings.php. Visit http://YOURSITE_COM/wikiadmin [YOURSITE_COM] and go through the mediawiki configuration again. Assuming you want the same settings, the only things you'll need to enter differently than last time are the "Wiki name" and "Database table prefix" settings.
Wiki name: Another Wiki
Admin username: WikiSysOp # Same as before
Password: YOUR_WIKI_ADMIN_PASSWORD # Same as before
Password confirm: YOUR_WIKI_ADMIN_PASSWORD # Same as before
Database type: MySQL # Same as before
Database host: localhost # Same as before
DB username: wikiuser # Same as before
DB password: YOUR_WIKI_DATABASE_USER_PASSWORD # Same as before
DB password confirm: YOUR_WIKI_DATABASE_USER_PASSWORD # Same as before
Database table prefix: another_

Click the "Install MediaWiki!" button. Then move LocalSettings.php to it's new home.

Edit the new LocalSettings-anotherwiki.php to change (or add) the same variables as before.

If you enabled uploading, don't forget to make the new directory, and let apache write to it.

Then update your apache config the same way you did earlier.

Ta-da! You just installed a new wiki. You can repeat those steps any time you want to create a new one.

You can now go add a user on one of your wikis, and that user will be able to log in to any of them. You can also log in at one wiki and go visit one of the others, and you will still be logged in as the same user.

Note that this doesn't affect things like "My Contributions" and whathaveyou. Those will still remain seperate for each wiki, as they should.

One final thing you should probably do is create a robots.txt in your DocumentRoot. I have one that looks like this:

Ok, I've got this up and running the way I want now, so I figure I'll document both for myself and for the greater good.

Goals:
* Run multiple mediawiki sites using a single codebase and sharing users and login sessions (log in once for all wikis)

Assumptions:
* A very basic understanding of the shell (commands like cd, mkdir, tar)
* You have at least a minimal understanding of mediawiki installation and can
  get one site up and running
* Apache 2 (though these methods may work with version 1, it's untested)
* Apache runs as group apache
* A single MySQL database for the wiki running on localhost.
* Root access to your webserver
* Latest version of mediawiki (1.13.2 at the time of this writing) tarballed in
  your home directory.
* Write access to /home/www (see Conventions) for your normal user account.
* You'll set up one wiki called "main" and create other wikis named whatever.
  If you don't want to have a "main" wiki, just change the naming of the
  first wiki to one of the wikis you would like to install.

Conventions:
* Paths will live under /home/www -- modify for your environment
* /home/www is NOT a DocumentRoot for apache (including any virtualhosts, etc)
* Commands will be inside quote tags and start with "$" for user commands and
  "#" for root commands
* Commands may be split over multiple lines for simplicity with a backslash (\).
* Text to go into files will also be inside quote tags, but will not be
  prefixed with $ or # (except for comments, it should be obvious which is
  which).
* Variables that you should change will be all caps and are self-documenting
  (e.g., YOUR_IP_ADDRESS)
* Database names, user names, etc can be altered to your liking. I'll just use
  what's conventional.

References:
* See the MediaWiki Installation Guide [mediawiki.org].

First, we'll set up two instances of mediawiki. One is kept pristine, and is just for running config/index.php to create all the database tables and LocalSettings.php. This wouldn't be totally necessary, but it makes for a nice way to do create new wikis without exposing yourself to nastiness during the config stage.

Give apache write access to the wikiadmin/config directory.

If you're paranoid (like me), you can remove the config files from the wikicode directory as we won't ever use them.

Now we have two installations of the code. One in the "wikicode" directory and one in the "wikiadmin" directory. For the sake of saving space, you can pare down the wikiadmin installation if you like. Personally, I removed everything under lanuages/messages except for MessagesEn.php. You can probably trim a lot more if you want.

It is absolutely critical that neither of these directories are configured as a DocumentRoot for your site, whether virtualhost or main configuration. Personally, I use an empty directory /home/www/wiki as my DocumentRoot for the wiki VirtualHost.

Next, edit your apache config to give you (and only you) access to your wikiadmin directory. If you are using a virtualhost, this goes inside the <VirtualHost> declaration, otherwise it can live in your main configuration:


Create your wiki database, and give the wiki access to it. You can do this however you like, but I prefer the simple CLI interface. Enter your mysql root user's password when prompted.

From here, you can set up your first wiki. Personally, I created a "Main Wiki." This first wiki setup will also defined the shared database tables, though any of them really could be used.

Go to http://YOURSITE_COM/wikiadmin [YOURSITE_COM]
Follow the configuration setup as normal. Most of the settings you can choose for yourself, I'll just mention a few important ones.

Wiki name: Main Wiki
Admin username: WikiSysOp
Password: YOUR_WIKI_ADMIN_PASSWORD
Password confirm: YOUR_WIKI_ADMIN_PASSWORD
Database type: MySQL
Database host: localhost
DB username: wikiuser
DB password: YOUR_WIKI_DATABASE_USER_PASSWORD
DB password confirm: YOUR_WIKI_DATABASE_USER_PASSWORD
Database table prefix: main_

And that's it. Make sure the rest of the settings are the way you want them, then click "Install MediaWiki!"

Next, move the LocalSettings.php you just created into the wikicode root directory. Be sure to rename it to LocalSettings-main.php! Also, be sure to move the file not just copy it. We don't want a LocalSettings.php hanging out in wikiadmin/config anymore.

Now you need to create the real LocalSettings.php. In your favorite text editor, just create /home/www/wikicode/LocalSettings.php to contain the following:

You can edit the html to present something nicer if you want. It really shouldn't happen that anyone sees that, unless you screw up your config.

Now you need to edit a few things in LocalSettings-main.php. These will be the things you change for each wiki you create. If these variables don't exist, add them.


If you want to enable uploads (and keep them seperate per wiki), then you'll first need to create an uploads directory and make it writeable by apache.

Then in LocalSettings-main.php change or create the variables:


Now we're all set. Update your apache config to serve the new wiki (in VirtualHost directive or main config).


Your first wiki is now ready to go. Visit http://YOURSITE_COM/main [YOURSITE_COM] to see it in action!

Now that we have all the difficult crap out of the way, adding new wikis is pretty simple. Let's say we're setting up a new wiki called "anotherwiki."

Go to the /wikiadmin site to create a new LocalSettings.php. Visit http://YOURSITE_COM/wikiadmin [YOURSITE_COM] and go through the mediawiki configuration again. Assuming you want the same settings, the only things you'll need to enter differently than last time are the "Wiki name" and "Database table prefix" settings.
Wiki name: Another Wiki
Admin username: WikiSysOp # Same as before
Password: YOUR_WIKI_ADMIN_PASSWORD # Same as before
Password confirm: YOUR_WIKI_ADMIN_PASSWORD # Same as before
Database type: MySQL # Same as before
Database host: localhost # Same as before
DB username: wikiuser # Same as before
DB password: YOUR_WIKI_DATABASE_USER_PASSWORD # Same as before
DB password confirm: YOUR_WIKI_DATABASE_USER_PASSWORD # Same as before
Database table prefix: another_

Click the "Install MediaWiki!" button. Then move LocalSettings.php to it's new home.

Edit the new LocalSettings-anotherwiki.php to change (or add) the same variables as before.

If you enabled uploading, don't forget to make the new directory, and let apache write to it.

Then update your apache config the same way you did earlier.

Ta-da! You just installed a new wiki. You can repeat those steps any time you want to create a new one.

You can now go add a user on one of your wikis, and that user will be able to log in to any of them. You can also log in at one wiki and go visit one of the others, and you will still be logged in as the same user.

Note that this doesn't affect things like "My Contributions" and whathaveyou. Those will still remain seperate for each wiki, as they should.

One final thing you should probably do is create a robots.txt in your DocumentRoot. I have one that looks like this:

How did I not know about vimperator [vimperator.mozdev.org] before? VIM keybindings and VIM-ish interface for freakin firefox!! No more <C-d> to go down a page and getting the stupid Add Bookmark dialog. I've only used it for a few minutes now and oh yes, this is sweet. Off to the documentation...

Going with the latest version also pushed me into moving to firefox 3 (which I've been meaning to do for a couple weeks now).

Oh, and did I mention the inline javascript and inline shell commands?! No? Yeah, this is so great.

I had a bit of a hitch working on my ulppd [sourceforge.net] project. Using Net::Server, it's not possible to chown the UNIX sockets it creates. You can put a chown into the post_bind_hook and it will be executed, but it's soon undone in post_bind (the next stage). This stage takes care of dropping permissions and chowning files to the user/group it's dropping permissions to.

With this patch, you can simply pass sock_owner and sock_group to the Net::Server, and the sockets will be chowned to that user/group respectively. If one or both options are omitted, the previous behavior is maintained.

Patch is available here [ulppd.svn.sourceforge.net]

Added a patch for the Net::Server documentation available here [ulppd.svn.sourceforge.net].

So I was getting all in a huff because logrotate wasn't rotating my log files (CentOS 5.2, but not a problem specific to that). It was really getting to me, as the config was EXACTLY the same as other servers (same OS, same versions of logrotate, even the same versions of crond running logrotate). I was testing by running "logrotate -d /etc/logrotate.conf" which reported that none of my logfiles needed to be rotated. The logfiles in question were over 2 weeks old and configured to rotate weekly, but still logrotate indicated they didn't need to be rotated. If I changed the config to daily instead of weekly, it correctly reported that they needed to be rotated.

Finally reading through the man page I noticed the -s, --state <statefile> flag, which I never really paid attention to (or knew about). Well, part of my investigation was trying to figure out how the heck logrotate determines when the log file was created. Turns out that it uses this state file to keep track of the last time logrotate rotated files, and thus when the files need to be rotated again.

That made the failure apparent -- I had only installed logrotate about 4 days ago, so on it's first run it wrote to the state file that the log files were "created" on that date. Since it only saw them as 4 days old, it determined they didn't need to be rotated. I confirmed that by editing the state file by hand to set the date back, and running logrotate with the -d flag again. Worked like a charm.

Long story short, after a fresh install and first run of logrotate it will take a full week to rotate weekly files, a month to rotate mothly files, etc. This can be corrected by editing the state file (/var/lib/logrotate.status by default) by hand.

So I was getting all in a huff because logrotate wasn't rotating my log files (CentOS 5.2, but not a problem specific to that). It was really getting to me, as the config was EXACTLY the same as other servers (same OS, same versions of logrotate, even the same versions of crond running logrotate). I was testing by running "logrotate -d /etc/logrotate.conf" which reported that none of my logfiles needed to be rotated. The logfiles in question were over 2 weeks old and configured to rotate weekly, but still logrotate indicated they didn't need to be rotated. If I changed the config to daily instead of weekly, it correctly reported that they needed to be rotated.

Finally reading through the man page I noticed the -s, --state <statefile> flag, which I never really paid attention to (or knew about). Well, part of my investigation was trying to figure out how the heck logrotate determines when the log file was created. Turns out that it uses this state file to keep track of the last time logrotate rotated files, and thus when the files need to be rotated again.

That made the failure apparent -- I had only installed logrotate about 4 days ago, so on it's first run it wrote to the state file that the log files were "created" on that date. Since it only saw them as 4 days old, it determined they didn't need to be rotated. I confirmed that by editing the state file by hand to set the date back, and running logrotate with the -d flag again. Worked like a charm.

Long story short, after a fresh install and first run of logrotate it will take a full week to rotate weekly files, a month to rotate mothly files, etc. This can be corrected by editing the state file (/var/lib/logrotate.status by default) by hand.

I've had a few goings-on recently on some projects...

I finally released my UniversaL Postfix Policy Daemon as opensource here [ulppd.sourceforge.net]. It's still in early alpha, though it works. I'm not sure if it'll be useful for anyone or not. It's basically a perl daemon and framework which makes it easy to implement arbitrary anti-spam stuff in perl. All you have to do is create a package that implements Net::Mail::Ulppd::Filter and tell the ulppd daemon to use it. There is absolutely no documentation yet, so if you're curious or would like to try it, go ahead and ask me questions. In fact, I haven't even packaged it for the downloads section yet, but you can grab it from sourceforge svn at svn://ulppd.svn.sourceforge.net/svnroot/ulppd [ulppd.svn.sourceforge.net]

I also put together some nifty stuff for managing DNS. Basically it's a package called Net::DNS::DynDomain implemented in a script called dnswatcher. It can watch dhcpd lease files, openvpn status files, local IP addresses, and static files, and automatically update internal/external servers' djbdns via Net::SSH::Perl when changes are detected. It's pretty localized to my environment, but it could easily be modified for your own stuff. If you're interested, once again let me know, and I can send you the source and help you set it up.

I've been pretty busy. In other news, I really should start working. Anyone need a sysadmin/perl developer (telecommute preferred and pay me lotsa dollars plskthx)?

Some updates...

ulppd now has a project page here [ulppd.projects.mifflinet.org]. You can check it out further there (not much up yet).

In addition, I have put out an alpha release of the software. It's in a working state, but I'm not ready to say it's beyond alpha yet.

When exporting LVM logical volumes, it is very important to use the fsid parameter in /etc/exports. Without that, you don't (necessarily) have consistent device numbers for the underlying devices, and you will see problems with "Stale NFS file handle" errors.

Since 0 (and "root") have special meaning in NFSv4, I start my fsid numbering at 100. That may or may not be necessary.

When exporting LVM logical volumes, it is very important to use the fsid parameter in /etc/exports. Without that, you don't (necessarily) have consistent device numbers for the underlying devices, and you will see problems with "Stale NFS file handle" errors.

Since 0 (and "root") have special meaning in NFSv4, I start my fsid numbering at 100. That may or may not be necessary.

There is very little documentation out there regarding mirroring with linux LVM. Mostly, it's just not necessary given that mirroring can be done at the device level instead. There are definitely some situations where it can be useful however. Here's an example situation (this is why I was interested):

VG: export
PV: /dev/xvdb1
PV: /dev/xvdc1 (this is the one that failed)

I have a volume group (named "export") with a number of (large) logical volumes. The volume group used to be on two physical volumes (underlying devices are raid), however one of those volumes failed. I was able to recover the raid and get all the databits moved off the physical volume, and reduced the volume group by that device. So now I have my volume group with a single PV (xvdb1), and have the raid back up and running (xvdc1). Before putting the physical volume back to normal use, I want to make sure the raid is stable. So what I'd like to do is:

1) Make sure that any new allocation takes place on the current raid device. I don't want to allocate on the new one until I know that it will stay up. It was a hairy recovery process the first time and I don't want to do it again.
2) Add the new raid back into the volume group, but use it only to create mirrors of the current LVs. Let it run like this for a couple weeks to see if the rebuilt raid stays ok.
3) If all goes well, break the mirror(s) leaving the data on the new raid, so I can drop the old one and re-create it as a hardware raid.
4) After that raid has been rebuilt, I want to add it back into the VG, and if possible reallocate the LVs so they stripe decently across the PVs.

I'll take care of the first two now, and the other two...hopefully all works out ok and I'll come back to post details about those.

1) making sure the current LVs stay on the current (only) PV -- To do so, I'll be mucking with the allocation policy. Long story short, the default policy (for PV) is "normal" which uses sensible values. For example, pieces of a mirror or stripes of an LV will not be allocated on the same PV. Makes sense. What I want to do is change the allocation policy from normal to a value of "cling". Cling means "stick to the physical volume that you were first allocated on." That's pretty simple to change with the lvconvert(8) command. I could change the PV allocation policy to cling, and since each LV has the (default) allocation policy "inherit", they would all start using the cling policy. I like to be difficult though, so I'll change it for each LV individually:



To just change the PV and let the LVs inherit it:


2) Add the new raid back in, and mirror existing logical volumes -- Not so tough; first add the PV back in (we know no extents will be allocated after the last step):



Now to use lvconvert to mirror the LVs. There are two important things to note. First, LVM wants to have 3 devices in order to do mirroring. There are two devices for each side of the mirror, and a third device is needed for the mirror log. The mirror log is a record on disk of what extents are synced between the mirror halves. In that way, if the server loses power or has some other failure, the mirror can be recreated and pick right back up where it was. There is another option: the mirror log can be kept in memory, however if the device is brought down and back up, it will need to be re-synced from the primary side of the mirror. For my purposes, I don't really care if the mirror breaks and never comes back, I just want to run it this way temporarily to see if one side fails. I'll use the "--mirrorlog core"

Going home from work -- to be continued...

There is very little documentation out there regarding mirroring with linux LVM. Mostly, it's just not necessary given that mirroring can be done at the device level instead. There are definitely some situations where it can be useful however. Here's an example situation (this is why I was interested):

VG: export
PV: /dev/xvdb1
PV: /dev/xvdc1 (this is the one that failed)

I have a volume group (named "export") with a number of (large) logical volumes. The volume group used to be on two physical volumes (underlying devices are raid), however one of those volumes failed. I was able to recover the raid and get all the databits moved off the physical volume, and reduced the volume group by that device. So now I have my volume group with a single PV (xvdb1), and have the raid back up and running (xvdc1). Before putting the physical volume back to normal use, I want to make sure the raid is stable. So what I'd like to do is:

1) Make sure that any new allocation takes place on the current raid device. I don't want to allocate on the new one until I know that it will stay up. It was a hairy recovery process the first time and I don't want to do it again.
2) Add the new raid back into the volume group, but use it only to create mirrors of the current LVs. Let it run like this for a couple weeks to see if the rebuilt raid stays ok.
3) If all goes well, break the mirror(s) leaving the data on the new raid, so I can drop the old one and re-create it as a hardware raid.
4) After that raid has been rebuilt, I want to add it back into the VG, and if possible reallocate the LVs so they stripe decently across the PVs.

I'll take care of the first two now, and the other two...hopefully all works out ok and I'll come back to post details about those.

1) making sure the current LVs stay on the current (only) PV -- To do so, I'll be mucking with the allocation policy. Long story short, the default policy (for PV) is "normal" which uses sensible values. For example, pieces of a mirror or stripes of an LV will not be allocated on the same PV. Makes sense. What I want to do is change the allocation policy from normal to a value of "cling". Cling means "stick to the physical volume that you were first allocated on." That's pretty simple to change with the lvconvert(8) command. I could change the PV allocation policy to cling, and since each LV has the (default) allocation policy "inherit", they would all start using the cling policy. I like to be difficult though, so I'll change it for each LV individually:



To just change the PV and let the LVs inherit it:


2) Add the new raid back in, and mirror existing logical volumes -- Not so tough; first add the PV back in (we know no extents will be allocated after the last step):



Now to use lvconvert to mirror the LVs. There are two important things to note. First, LVM wants to have 3 devices in order to do mirroring. There are two devices for each side of the mirror, and a third device is needed for the mirror log. The mirror log is a record on disk of what extents are synced between the mirror halves. In that way, if the server loses power or has some other failure, the mirror can be recreated and pick right back up where it was. There is another option: the mirror log can be kept in memory, however if the device is brought down and back up, it will need to be re-synced from the primary side of the mirror. For my purposes, I don't really care if the mirror breaks and never comes back, I just want to run it this way temporarily to see if one side fails. I'll use the "--mirrorlog core"

Going home from work -- to be continued...

This should be compatible with any controller supported by tw_cli.

#!/bin/bash

TW_CLI="/YOUR_PATH_TO/tw_cli" # change this for your environment
RAID_CHECK_CMD="$TW_CLI /c0 show allunitstatus"
RAID_FULLSTATUS_CMD="$TW_CLI /c0 show all"
EMAIL_TO="YOUR_EMAIL@DDRESS" # change this for your environment
EMAIL_FROM="FROM_EMAIL@DDRESS" # change this for your environment
SENDMAIL="/usr/sbin/sendmail -f $EMAIL_FROM"
TOUCH="/bin/touch"
SEMAPHORE="/root/.raidstatus_mailed" # change this if you like
HOSTNAME=$(hostname -f) # THIS IS NOT CROSS-PLATFORM SAFE - should be OK for any linux, otherwise remove the -f
INTERVAL=600

while : ; do
 if $RAID_CHECK_CMD |grep -q 'Not Optimal Units = [^0]'; then # Scream OH SHIT and run.
  # Been an hour since I emailed last?
  if [ ! \( -r $SEMAPHORE \) -o \( $(($(date +%s) - $(date +%s -r $SEMAPHORE))) -gt 3600 \) ]; then
   FULLSTATUS="$($RAID_FULLSTATUS_CMD)"
   $SENDMAIL $EMAIL_TO <<EOM
From: "3ware 9650SE on $HOSTNAME" <$EMAIL_FROM>
To: <$EMAIL_TO>
Subject: RAID problem detected on $HOSTNAME

$0 has detected a problem

Full controller status:

$FULLSTATUS

Please resolve this ASAP.
EOM
   touch $SEMAPHORE
  fi
 fi
 sleep $INTERVAL
done

This should be compatible with any controller supported by tw_cli.

#!/bin/bash

TW_CLI="/YOUR_PATH_TO/tw_cli" # change this for your environment
RAID_CHECK_CMD="$TW_CLI /c0 show allunitstatus"
RAID_FULLSTATUS_CMD="$TW_CLI /c0 show all"
EMAIL_TO="YOUR_EMAIL@DDRESS" # change this for your environment
EMAIL_FROM="FROM_EMAIL@DDRESS" # change this for your environment
SENDMAIL="/usr/sbin/sendmail -f $EMAIL_FROM"
TOUCH="/bin/touch"
SEMAPHORE="/root/.raidstatus_mailed" # change this if you like
HOSTNAME=$(hostname -f) # THIS IS NOT CROSS-PLATFORM SAFE - should be OK for any linux, otherwise remove the -f
INTERVAL=600

while : ; do
 if $RAID_CHECK_CMD |grep -q 'Not Optimal Units = [^0]'; then # Scream OH SHIT and run.
  # Been an hour since I emailed last?
  if [ ! \( -r $SEMAPHORE \) -o \( $(($(date +%s) - $(date +%s -r $SEMAPHORE))) -gt 3600 \) ]; then
   FULLSTATUS="$($RAID_FULLSTATUS_CMD)"
   $SENDMAIL $EMAIL_TO <<EOM
From: "3ware 9650SE on $HOSTNAME" <$EMAIL_FROM>
To: <$EMAIL_TO>
Subject: RAID problem detected on $HOSTNAME

$0 has detected a problem

Full controller status:

$FULLSTATUS

Please resolve this ASAP.
EOM
   touch $SEMAPHORE
  fi
 fi
 sleep $INTERVAL
done

LISTEN. VERY. CLOSELY.

The phrase "page cannot be displayed" is not a problem description.

How can anyone in any sort of technical field (web design, no less) ever think that that means anything at all?! "What's the problem?" "Page cannot be displayed."


Oh.....


.....


I see.....


.....


.....


.....you're an idiot.

LISTEN. VERY. CLOSELY.

The phrase "page cannot be displayed" is not a problem description.

How can anyone in any sort of technical field (web design, no less) ever think that that means anything at all?! "What's the problem?" "Page cannot be displayed."


Oh.....


.....


I see.....


.....


.....


.....you're an idiot.

Assume you have a repository in file:///home/repos/my_repo on box1. Also assume there are two projects inside: /proj1 and /proj2.

Say on box2 we have a repository in file:///home/repos/other_repo, and we'd like to move proj2 over there. The following will do so:

[svnuser@box1]$ svnadmin dump /home/repos/my_repo |svndumpfilter include proj2 >proj2.dump

I'm going to move my project from /proj2 on box1's repo to /proj2 on box2's repo, so the Node-paths will stay the same. If I was renaming it to say /newproj2 on box2's repo, at this point I'd run:

[svnuser@box1]$ sed -i 's/Node-path: proj2/Node-path: newproj2/g' proj2.dump

Either way, then move proj2.dump over to box2, and on box2:

[svnuser@box2]$ svnadmin load /home/repos/other_repo < proj2.dump

That's going to run through every transaction your project has gone through, so it will take a little while. Keep this in mind when making your dump as well...it's going to take up storage for every revision of your project. That can be an issue on more mature projects.

But, there you have it. The project is now available on box2 at file:///home/repos/other_repo/proj2.

Assume you have a repository in file:///home/repos/my_repo on box1. Also assume there are two projects inside: /proj1 and /proj2.

Say on box2 we have a repository in file:///home/repos/other_repo, and we'd like to move proj2 over there. The following will do so:

[svnuser@box1]$ svnadmin dump /home/repos/my_repo |svndumpfilter include proj2 >proj2.dump

I'm going to move my project from /proj2 on box1's repo to /proj2 on box2's repo, so the Node-paths will stay the same. If I was renaming it to say /newproj2 on box2's repo, at this point I'd run:

[svnuser@box1]$ sed -i 's/Node-path: proj2/Node-path: newproj2/g' proj2.dump

Either way, then move proj2.dump over to box2, and on box2:

[svnuser@box2]$ svnadmin load /home/repos/other_repo < proj2.dump

That's going to run through every transaction your project has gone through, so it will take a little while. Keep this in mind when making your dump as well...it's going to take up storage for every revision of your project. That can be an issue on more mature projects.

But, there you have it. The project is now available on box2 at file:///home/repos/other_repo/proj2.

I hit a stupid bug that should not have been happening (I didn't mean for this to be included in my kernel). When patching and rebuilding kernels to fix an awfully scary local exploit [userlame.com], I left cpu frequency scaling enabled including the powernow-k8 governor. These were domain 0 kernels for xen VM hosts, and when under heavy load in the domU, it would hit a general protection fault which apparently comes from this governor. Here's a couple examples of hitting the bug:





And at that point, I'd have to xm destroy the domains and re-create them. Some googling of "general protection fault" and "vma_prio_tree_remove" pointed directly at powernow-k8. I went and checked the dom0 kernels, and sure enough CPU_FREQ was enabled and powernow-k8 was in there.

Recompiled and about to restart the machines. I really hope this helps. The domU is reproducibly crashing when running an emerge and it really sucks. It crashes and dumps stack trace from different programs including so far sed, ebuild.sh and x86_64-pc-linux.

Looks like this wasn't actually caused by powernow. I'm still able to reproduce it pretty consistently after removing CPU_FREQ entirely.

However, I am completely dumb and found some new information that helps a lot. It looks like this is only occurring on one of my VM hosts. That is some great freakin news. That means I can get some memtest86 going and track down the bad memory (hopefully).

I think I lucked out here.

To exit vim with an error code, use :cq.

To exit vim with an error code, use :cq.

The title says it all. I've pondered this [plasmasturm.org] myself many times, and always reached the same conclusion.

The title says it all. I've pondered this [plasmasturm.org] myself many times, and always reached the same conclusion.

Um, turns out there's a local root exploit in the linux kernel versions 2.6.17 to 2.6.24.1. I just tried it on my machines - yeah, it works. Read the slashdot discussion here [it.slashdot.org] and see the proof of concept code here [milw0rm.com].

Still reading about it now. This is huge. Hold me.

Never posted back about this did I? Oops. Well, here's a fix that should apply to most any vulnerable kernel. The affected file is fs/splice.c:

After months of rigorous testing, sweat, blood and tears I've finally perfected the formula.

Here it is.

Are you ready?

Hrm...

You did post in the very thread about the dates [userlame.com].

Long story short, as of now I dunno what the deal is.

Why the hell didn't I know about this [postgresql.org] before?! Right now I do some funky conversion from ipv4 -> 32 bit integer so I can do binary math for netmasks an such. I went to find a way I could store ipv6 addresses, and there's that looking me in the face.

Got to look into that more.

Functions:

http://www.postgresql.org/docs/8.0/static/functions-net.html

The title says it all. There are some very big things in the works. That's all I have time to say now. The big things are keeping me busy.

Oh, and there was some breakage around the site. That's all better. Oops.

I'm developing and building out some really really cool nerd-stuffs right now.

The hardware list in my new stuff (mostly accurate) is available: here [userlame.com].

Also I apparently substitute emoticons before links - a link with "colon capital O" in it breaks horribly.

I don't like yahoo very much these days. They are (were) a minor but annoying thorn in my side. Lemme splain.

Yahoo has this nice feature for its email client called a vacation response. Basically you type in what you want it to say - "out of the office riding goats" or whathaveyou, and when someone sends an email to your account it sends back an email with the subject "Yahoo! Auto Response" - the text of the email contains "out of the office riding goats" or whatever you typed in.

Great. I can tell people I'm not going to read their email for a while. Whoopie.

Some flaws (besides the obvious AUTO-RESPONSES ARE EVIL. DO NOT USE THEM):

1) It doesn't keep track of who it has sent a response to. If I sent two emails to an account with that activiated, I'd get two "Yahoo! Auto Response" replies. Not terrible, though it could possibly lead to some funny mail loops.

2) It's really easy to set up a yahoo account. It would be pretty easy to script an account setup that would present just the captchas to be solved. Then the script could set up an auto-response.

3) No anti-spam policies at all seem to be applied to email that will be auto-responded to. At least I know they aren't checking SPF.

Mix em all together and you get a delicious spam pie. I've written a proof of concept (not a script, you'll have to do it by hand - the rest is an exercise I'll kill the reader for following) so you can see it in action. Try it yourself [userlame.com]. But please, have some netiquette.

In short, spammers are sending email to fake accounts with spam vacation responses. They are faking lots and lots of from addresses on those emails and the spam vacation response is sent to the faked from address. Thus, I end up with a buttload of spam.

It also is completely valid mail. It comes from otherwise (I guess) sane outbound mailservers for yahoo. The email will have a valid domain DomainKey-Signature and everything. It was getting by all my anti-spam because it looks so real.

And so finally, a resolution...spamassassin. Here's a simple rule to send all that crap where it belongs. Adjust the score accordingly if you need to.

header YAHOO_AUTORESPONSE_SPAMZ Subject =~ /^Yahoo! Auto Response$/
score YAHOO_AUTORESPONSE_SPAMZ 5.0
describe YAHOO_AUTORESPONSE_SPAMZ Yahoo auto-response spam

I'm going to bed.

Edit: Yikes, that link was broken. Originally it was .html, and I moved it to .php but didn't update the link. My browser had the .html cached so I never noticed. Sigh.

Well well, it actually just got much easier [internetcommunications.tmcnet.com] to abuse this.

Just so I won't forget and maybe google will spider this and it'll help somebody. There's a (kinda) bug [bugs.gentoo.org] in gentoo's 2007.0 stage3. I'm using amd64, but it's due to the version of ssh which was put into the stage3, so I'm sure it's across platforms. /etc/skel/.ssh is created with permissions 0600 - so when you add a user, by default they won't be able to cd to ~/.ssh. That's a problem when using ssh keys and it frustrated me for a few minutes because I couldn't login in to new servers.

Fix by the following:


After this any new users can use their .ssh directory. If you've already added users and are having problems here's a quick fix:

Just so I won't forget and maybe google will spider this and it'll help somebody. There's a (kinda) bug [bugs.gentoo.org] in gentoo's 2007.0 stage3. I'm using amd64, but it's due to the version of ssh which was put into the stage3, so I'm sure it's across platforms. /etc/skel/.ssh is created with permissions 0600 - so when you add a user, by default they won't be able to cd to ~/.ssh. That's a problem when using ssh keys and it frustrated me for a few minutes because I couldn't login in to new servers.

Fix by the following:


After this any new users can use their .ssh directory. If you've already added users and are having problems here's a quick fix:

I meant to post this a while ago, but never got around to it. If you've purchased a ZyXEL GS2024, and it came with firmware V3.8(LT.0)C0 I pity you. I too bought this lovely lemon. Long story short, it sucked. I was really disappointed when I had my new switch running for about 5 minutes and thought I had to RMA it.

I pulled it out of the box, fired it up, and started playing with it, and no sooner had I upped the serial baud rate when the hardware error light came on. Then it started flashing. Not steadily either, just random on/off. The second command I checked out was reading the logs. I found a whooole bunch of errors in the log about the fan dropping below threshold and recovering (many times per second). From running the diagnostics on the switch the fan was peachy. Sigh.

I checked for new firmware, but this was the most recent. So recent in fact (I thought) that it wasn't even on the manufacturer's site. The manual for 3.8 was there and released only weeks before. I'm sure they actually pulled the firmware down when they realized what a bad bug it had. I googled around and found someone else complaining of the same errors after upgrading from 3.6 to 3.8. I can't find the site now, which is weird. I was there numerous times and posted on it. Hm. I'd post the log messages from the switch, but mine are gone and the only place I knew that had them was that site.

Anyway, I downgraded the firmware and have been playing around with it while running version 3.6. The only feature that I noticed was missing was the ability to use ssh keys. I also noticed logging in as admin wasn't automatically in enable mode in the earlier version. Whatever.

Well, earlier this month, ZyXEL put out a bugfixed version of 3.8 which is available on their site [us.zyxel.com]. If you were one of the lucky ones to get your GS-2024 shipped with firmware V3.8(LT.0)C0 - well, hopefully you've downgraded, but now you can go get a better (read: working) version of 3.8. The version name is very similar - V3.8(LT.1)C0

I would like to point out that it's a very nice switch and I'm quite pleased with it now. ZyXEL was very fast with a fix, but it was disappointing that a bug so painfully obvious was put out, and loaded on production models. Running it for more than a minute or two showed the problem.

Update: Found the forum post I was talking about here [fixya.com]. The fan log messages looked like:

I meant to post this a while ago, but never got around to it. If you've purchased a ZyXEL GS2024, and it came with firmware V3.8(LT.0)C0 I pity you. I too bought this lovely lemon. Long story short, it sucked. I was really disappointed when I had my new switch running for about 5 minutes and thought I had to RMA it.

I pulled it out of the box, fired it up, and started playing with it, and no sooner had I upped the serial baud rate when the hardware error light came on. Then it started flashing. Not steadily either, just random on/off. The second command I checked out was reading the logs. I found a whooole bunch of errors in the log about the fan dropping below threshold and recovering (many times per second). From running the diagnostics on the switch the fan was peachy. Sigh.

I checked for new firmware, but this was the most recent. So recent in fact (I thought) that it wasn't even on the manufacturer's site. The manual for 3.8 was there and released only weeks before. I'm sure they actually pulled the firmware down when they realized what a bad bug it had. I googled around and found someone else complaining of the same errors after upgrading from 3.6 to 3.8. I can't find the site now, which is weird. I was there numerous times and posted on it. Hm. I'd post the log messages from the switch, but mine are gone and the only place I knew that had them was that site.

Anyway, I downgraded the firmware and have been playing around with it while running version 3.6. The only feature that I noticed was missing was the ability to use ssh keys. I also noticed logging in as admin wasn't automatically in enable mode in the earlier version. Whatever.

Well, earlier this month, ZyXEL put out a bugfixed version of 3.8 which is available on their site [us.zyxel.com]. If you were one of the lucky ones to get your GS-2024 shipped with firmware V3.8(LT.0)C0 - well, hopefully you've downgraded, but now you can go get a better (read: working) version of 3.8. The version name is very similar - V3.8(LT.1)C0

I would like to point out that it's a very nice switch and I'm quite pleased with it now. ZyXEL was very fast with a fix, but it was disappointing that a bug so painfully obvious was put out, and loaded on production models. Running it for more than a minute or two showed the problem.

Update: Found the forum post I was talking about here [fixya.com]. The fan log messages looked like: