Home > Casper, Linux > Fixing Casper’s MySQL database with mysqlcheck

Fixing Casper’s MySQL database with mysqlcheck

After some VMware host issues that were out of my control, the RHEL VMs that my production and test Casper servers are hosted on were unexpectedly rebooted a couple of times. When I checked the VMs afterwards, everything appeared to be OK. I figured that I had been fortunate, until my Casper test server sent me the nightly “Successful backup of the Casper database” email and my production server didn’t.

Uh oh.

When I checked the directory when my production server stores its backups of the Casper database, there wasn’t a backup from the night before. I immediately launched the JAMF Database Utility application and had it make a backup of the production database. A task which normally would take 10 minutes or so now took 40 minutes.

Not good.

To lighten the load on the database, I went into the JSS and had it manually flush all but the last week’s worth of logs (I normally retain 30 days of logs.)

Screen Shot 2014-02-01 at 5.12.10 PM

Screen Shot 2014-02-01 at 5.12.50 PM

Once the log flush had completed, I then rebooted the box. On reboot, the JSS initialized and then hung halfway through the JSS startup process.

Screen Shot 2014-02-01 at 4.53.48 PM

Really not good.

For the details of how I fixed this, see below the jump.

Casper’s JSS server software comes in two parts, a Tomcat web application and a MySQL database named jamfsoftware. As far as I could tell, the Tomcat part was OK but was having problems communicating with Casper’s jamfsoftware MySQL database.

To ensure that my Casper-managed Macs were not trying to update the database while I was working on fixing it, I stopped the Tomcat web application by running the following command as root:

/etc/rc.d/init.d/jamf.tomcat7 stop

Screen Shot 2014-02-01 at 5.01.31 PM

I then restarted MySQL with the following command:

/etc/init.d/mysqld restart

Screen Shot 2014-02-01 at 5.04.17 PM

After the MySQL restart, I ran the following mysqlcheck command to repair the database:

mysqlcheck -repair jamfsoftware -u root -p

Screen Shot 2014-02-01 at 5.22.34 PM

In this case, mysqlcheck was set to repair all of the tables in the jamfsoftware database.

90 minutes later, the repair process had finished checking and repairing the database. I then ran optimization with mysqlcheck using the following command:

mysqlcheck -o jamfsoftware -u root -p

Screen Shot 2014-02-01 at 5.23.17 PM

Since the database tables had just been repaired, optimization finished within seconds. I then rebooted the server. I could have just restarted the Tomcat web application, but I wanted to make sure everything worked on restart.

Screen Shot 2014-02-01 at 5.09.32 PM

On restart, the Casper service started up normally. Later that night, I once again received the nightly “Successful backup of the Casper database” email from both my test and production servers.

Screen Shot 2014-02-01 at 4.59.57 PM

Categories: Casper, Linux
  1. Sergio Aviles
    November 24, 2014 at 5:53 pm

    Thanks for this. I experienced this just this past Friday and your post here got me through. I really appreciate all the pioneering and trailblazing work you do for the community.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: