Rotate and Compress Application Logs on a Schedule

Application logs grow silently and can fill a disk within days on busy servers, causing outages or dropped writes. Running log rotation on a schedule ensures old logs are compressed, archived, and pruned before they become a problem. This recipe shows you how to set up a reliable nightly log rotation job and monitor it with a CronJobPro heartbeat so you know it actually ran and succeeded.

Schedule

0 2 * * *

Every day at 2:00 AM server time

Setup

  1. 1

    Verify or install logrotate

    On Debian/Ubuntu run: apt-get install -y logrotate. On RHEL/CentOS: yum install -y logrotate. Most Linux distributions ship it by default. Confirm with: logrotate --version.

  2. 2

    Create or review your logrotate config

    Drop a config file at /etc/logrotate.d/myapp (see example in the script section). Key directives: daily or weekly, rotate N (how many old copies to keep), compress and delaycompress (gzip all but the most recent rotated file), missingok (skip silently if log is absent), notifempty (skip empty logs), postrotate/endscript (send a signal to your app to reopen its log file handle).

  3. 3

    Create the wrapper shell script

    Save the script below to /usr/local/bin/rotate-app-logs.sh and make it executable: chmod +x /usr/local/bin/rotate-app-logs.sh. The script runs logrotate, captures its exit code, pings the CronJobPro heartbeat on success, or reports failure on error.

  4. 4

    Add the cron entry

    Run: crontab -e (as root or the appropriate user) and add the line: 0 2 * * * /usr/local/bin/rotate-app-logs.sh >> /var/log/rotate-app-logs-cron.log 2>&1. This runs every day at 2 AM and appends stdout and stderr to a local audit log.

  5. 5

    Create a CronJobPro heartbeat monitor

    In your CronJobPro dashboard create a new Heartbeat monitor. Set the period to 24 hours and the grace period to 30 minutes (giving the job until 2:30 AM to ping in). Copy the generated URL (https://cronjobpro.com/ping/<token>) and paste it into your script where indicated. Configure alert channels such as email or Slack so you are notified immediately if the ping does not arrive.

The script

bash

#!/usr/bin/env bash
# /usr/local/bin/rotate-app-logs.sh
# Runs logrotate for your application and pings a CronJobPro heartbeat on success.
# Edit LOGROTATE_CONF and HEARTBEAT_URL before use.

set -euo pipefail

LOGROTATE_CONF="/etc/logrotate.d/myapp"
HEARTBEAT_URL="https://cronjobpro.com/ping/YOUR_TOKEN_HERE"
STATE_FILE="/var/lib/logrotate/myapp.status"
LOG_TAG="rotate-app-logs"

log() {
  echo "[$(date -u '+%Y-%m-%dT%H:%M:%SZ')] $*"
}

log "Starting log rotation for config: ${LOGROTATE_CONF}"

# Run logrotate verbosely; --state keeps rotation state separate from system default
if logrotate --verbose --state "${STATE_FILE}" "${LOGROTATE_CONF}" 2>&1; then
  log "logrotate completed successfully"
  EXIT_CODE=0
else
  EXIT_CODE=$?
  log "logrotate exited with code ${EXIT_CODE}"
fi

# Report outcome to CronJobPro
if [ "${EXIT_CODE}" -eq 0 ]; then
  # Success ping: tells CronJobPro the job ran and finished cleanly
  curl --silent --max-time 10 --retry 2 "${HEARTBEAT_URL}" > /dev/null
  log "Heartbeat ping sent to CronJobPro"
else
  # Failure ping: immediate alert without waiting for the period to expire
  curl --silent --max-time 10 --retry 2 "${HEARTBEAT_URL}/fail" > /dev/null
  log "Failure ping sent to CronJobPro (exit code ${EXIT_CODE})"
  exit "${EXIT_CODE}"
fi

# ---------------------------------------------------------------
# Example /etc/logrotate.d/myapp (adjust paths and directives)
# ---------------------------------------------------------------
# /var/log/myapp/*.log {
#     daily
#     rotate 14
#     compress
#     delaycompress
#     missingok
#     notifempty
#     create 0640 www-data adm
#     sharedscripts
#     postrotate
#         # Tell your app to reopen its log file (example: nginx)
#         # kill -USR1 $(cat /run/myapp/myapp.pid) 2>/dev/null || true
#     endscript
# }
# ---------------------------------------------------------------

Monitor it

Create a Heartbeat monitor in CronJobPro and set the expected period to 24 hours with a 30-minute grace window to account for server load at 2 AM. The script calls https://cronjobpro.com/ping/YOUR_TOKEN on a clean logrotate exit and https://cronjobpro.com/ping/YOUR_TOKEN/fail if logrotate returns a non-zero exit code, so CronJobPro can alert you immediately on failure rather than waiting for the missed-ping timeout. If the ping never arrives at all — for example because cron itself failed to fire, the server was down, or the script crashed before reaching the curl call — CronJobPro will alert you after the grace period expires. Connect your preferred alert channel (email, Slack, Discord, PagerDuty, or a webhook) in the monitor settings so the right person is notified without having to check dashboards manually.

Frequently asked questions

What is the difference between running logrotate via cron and the system logrotate cron job?

Most distributions install a system-wide /etc/cron.daily/logrotate entry that processes all configs in /etc/logrotate.d/. Running your own cron entry gives you explicit control over timing, a separate state file per application, and the ability to add a heartbeat ping so you know it ran. If you prefer to rely on the system cron, you can still add the heartbeat ping inside a postrotate script in your logrotate config, though you lose the fail-ping on logrotate errors.

Why use delaycompress instead of compress alone?

compress gzips the file immediately on rotation, but some applications keep an open file descriptor on the just-rotated file and will fail to write if the name changes and the content is compressed. delaycompress waits one rotation cycle before compressing, so the previous log is still readable in plain text during the next day's run. Use compress alone only when you are certain your application reopens its log file promptly after rotation (typically via a postrotate signal).

How many rotated copies should I keep?

A common default is 14 daily copies (two weeks) for application logs and 4 weekly copies for heavier logs like database slow-query logs. Calculate your average log growth per day, multiply by the retention count, and confirm that fits within your disk budget with room to spare. The rotate directive in your logrotate config controls this number directly.

The heartbeat ping fails sometimes because the server has no outbound internet access. What can I do?

If your server is behind a firewall or in a private network, ensure outbound HTTPS to cronjobpro.com on port 443 is allowed. Alternatively, proxy the ping through an internal relay, or schedule the job on a machine that does have outbound access and use SSH to trigger logrotate remotely. The /ping/<token>/exitcode/<n> endpoint also lets you report a numeric exit code if you prefer a single endpoint for both success (0) and failure scenarios.

More recipes

Rotate and Compress Application Logs on a Schedule | CronJobPro