Fixing awstats after a Plesk 8.3 migration

Updated Mar 06


Note!

I've gotten a few bug reports from people, so let me describe one issue you may see. This script is hard-coded to run stats against /var/www/vhosts/[domain]/statistics/logs/access_log.processed. This is obviously not the best solution, and it creates issues for people who are using log rotation since they may have logs called access_log.processed.1, etc. I am working on a solution in addition to rewriting large swaths of my code for better robustness and error checking. I will be posting a new version within the next two weeks, stay tuned.

I didn't expect this script to become popular at all and I really appreciate all the bug reports people have been sending in, thanks everybody.

Overview

Several problems with statistics have been reported when moving from Plesk 8.2 to Plesk 8.3. The solution is to completely rebuild the generated awstats information. This is very time-consuming, especially if you have many sites with statistics going back six months or more. In addition to generating stats information from the Apache access logs, you need to manually generate the HTML files for each month's stats.

After spending an hour and a half fixing this issue, I wrote a doo-dad which does it for you. Here is what my script does:

- Reads a list of domains on your server
- Backs up the access logs and processed access logs for each domain to /root/log_backups/
- Rebuilds statistics and generates new HTML pages

New version notes!

Version 1.2 is the newest version, updated on Feb 19. It contains a few minor bug fixes, some additional error checking, and it no longer backs up your access logs, since it is not touching them. I had a report from a guy who had 800M of logs so duplicating them took a bit of time and also wasted some space on his server. Incidentally, the script did resolve his awstats issue and took two hours to run against 30 domains with stats going back to June 2007. The new version is also gzipped because er... I think I'm supposed to do it that way. :)

Usage

The script consists of two components: fixstats.pl and awstats_makemonth.sh. fixstats.pl is the main script which contains most of the logic, while awstats_makemonth.sh handles teling awstats to generate monthly reports. awstats_makemonth.sh was written by Europheus of The Internet.

To install the script, first download it to your server and place it in /root:

cd /root
wget http://nickforsale.com/fixstats/fixstats.tar.gz


Then, extract the files from the archive:

tar -xzf fixstats.tar.gz


Finally, run the script to display usage information: 

# ./fixstats.pl
fixstats v1.2
Nick Lewis <nick.lewis@gmail.com>

Usage: ./fixstats.pl [--all | --domain= | --verbose | --test | --help]

--all           : Run statistics on all domains. May not be used in conjunction with --domain.
--domain        : Run statistics on specified domain only. May not be used in conjunction with --all.
--verbose       : Verbose mode. Displays domains as they are processed.
--test          : Test mode. Does everything except call awstats. For debugging purposes mostly.
--help          : This message.


Usage is pretty self-explanatory. Note that either -all or --domain=<domain> is required. I may drop the -all switch and make it the default behavior if this is confusing to anybody.

If you have a lot of domains or stats going back several months / years it will take a long time to run. On my server, with eight domains, three of which have stats enabled, it takes six minutes to process the two months or so of stats. It's a lot faster than doing everything manually.

Notes

In verbose mode, you will see every domain listed for the steps "Generating initial stats...", "Generating complete stats..." and "Linking monthly reports...". However for the step "Building monthly reports..." the only domains listed will be the ones which have AWstats enabled in Plesk. This is normal.

The script files MUST be placed in /root.

This ONLY fixes statistics for awstats, it does not do anything for Webalizer.

Any questions, comments, bugs please email nick.lewis@gmail.com