There are already several descriptions of Nginx, but I think my configuration will also be interesting. 
The situation is as follows: there is a website hosted on several IIS servers (online store), with a balancer in front of it. Between them, it was decided to install nginx to reduce the load on IIS. 

The bulk of dynamic content is displayed by Ajax, so caching product pages is quite safe.

Plus, I want to maintain the validity of popular pages in the cache automatically. 

So, first install fresh nginx – without it will not work. We also need wget and curl.
I will not describe in detail the configuration of the proxy itself, but will focus in detail on the methods of keeping the cache up to date.

Nginx configuration itself:

worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
proxy_cache_path /var/cache/nginx levels=2:2 keys_zone=STATIC:512m
inactive=24h max_size=32g;

sendfile on;
keepalive_timeout 65;

gzip on;
gzip_proxied any;
gzip_min_length 1100;
gzip_http_version 1.0;
gzip_buffers 4 8k;
gzip_comp_level 9;
gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript application/json;

server {
server_name 127.0.0.1;
listen 80;

set $backend 127.0.0.2;

log_format cache ‘$remote_addr|$time_local|$request_uri|$request_method|$status|$http_referer|$cookie___sortOrder|$IsAuth|$sent_http_content_type|$http_user_agent’;
access_log /var/log/nginx/proxy_access.log cache;
error_log off;

location ~* /(basket.aspx|visitedgoods.aspx|users/|sale/order.aspx|sale/posted.aspx) {

proxy_pass http://$backend;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
proxy_redirect off;
set $IsAuth 1;
if ($cookie_AUTH = “”) {
set $IsAuth 0;
}
}
location / {
if ($args ~* (.*)\|(.*)) {
set $brand $1$2;
rewrite ^(.*) $1?$brand? redirect;
}
if ($args ~* (.*)\%7C(.*)) {
set $brand $1$2;
rewrite ^(.*) $1?$brand? redirect;
}
rewrite ([a-zA-Z0-9]+)\|([a-zA-Z0-9]+) $1$2? permanent;
rewrite ([a-zA-Z0-9]+)\%7C([a-zA-Z0-9]+) $1$2? permanent;
rewrite (.*)\%7C$ $1? permanent;
rewrite (.*)\| $1? permanent;
proxy_pass http://$backend;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass_header Set-Cookie;
proxy_ignore_headers “Expires” “Cache-Control” “Set-Cookie”;

set $IsAuth 1;
if ($cookie_AUTH = “”) {
set $IsAuth 0;
}
set $DoBypass 0;
if ($http_user_agent = “WGET-POST-daemon”) {
set $DoBypass 1;
}
proxy_cache STATIC;
proxy_cache_key “$host$uri$is_args$args $cookie___sortOrder $IsAuth”;
proxy_cache_valid 200 301 302 304 30m;
proxy_cache_bypass $DoBypass;
proxy_cache_use_stale error timeout invalid_header updating;
proxy_connect_timeout 100ms;
proxy_redirect off;
}
}
}

So, the proxy is ready. Now the fun part. 

We want to have in the cache TOP-20000 pages for the last day. The log looks like this: Logs are turned every hour and stored for 24 hours. Unfortunately, logrotate cannot convert logs less than once a day, so the average hack is applied: size 1 to the configuration file and logrotate -f /etc/nginx.rotate per crown once an hour. Script to create a list of the most visited pages:

127.0.0.3|20/Oct/2011:15:45:43 +0400|/catalog/25/women.aspx|GET|200|http://127.0.0.1/|-|0|text/html; charset=windows-1251|Opera/9.80 (Windows NT 6.1; U; ru) Presto/2.9.168 Version/11.51

#!/bin/bash 
ourIP=$(ifconfig | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}')

curl  -H “Cookie: BasketID=KYKY-B-PYKY” -H “Content-Type: application/json” -d “{\”login\”:\”USERNAME\”,\”password\”:\”PASS\”,\”remeberMe\”:\”true\”}” -c /var/log/nginx/cookies1.txthttp://127.0.0.1/Resources/Services/SystemService.asmx/SignIn

line=$(tail -n 1 /var/log/nginx/cookies1.txt)
AUTH=$(echo $line | awk '{wild=$7; print wild}')
cp /var/log/nginx/proxy_access.log /var/log/nginx/overall_proxy

FILES=/var/log/nginx/*.gz
for f in $FILES
do
cat $f | gunzip>> /var/log/nginx/overall_proxy

done
rm -f /var/log/nginx/wget/*
awk -v ourIP="$ourIP" '{ FS="|"; ip = $1; url = $3; code = $5; catsrt=$7; isauth=$8; ct=$9; if (ip != ourIP) if (url !~ "basket.aspx" && url !~ "visitedgoods.aspx" && url !~ "users/" && url !~ "sale/order.aspx" && url !~ "sale/posted.aspx") if (code = "200") if (ct ~ "text/html;") print "http://" ourIP url "|" catsrt "|" isauth}' / var/log/nginx/overall_proxy | sort | uniq -c | sort -n -k1,6 | tail -n20000 | awk '{print $2}' | awk -vAUTH="$AUTH" '{FS="|"}$2=="-"{$2=""} $3=="0"{$3=""} $3=="1"{$3=AUTH}{print "-b --header=\"Cookie: __sortOrder="$2"; AUTH="$3"\" -o /dev/null -O /dev/null "$1 }'> /var/log/nginx/cache.dat
 rm -f /var/log/nginx/overall_proxy
cd /var/log/nginx/wget

split -l 2000 /var/log/nginx/cache.dat
rm -f /var/log/nginx/cache.dat


So, at the output we get files with commands: 
Then every 20 minutes we have to go through this list and request each of the pages from the server to validate the cache.

-b --header="Cookie: __sortOrder=; AUTH=" -o /dev/null -O /dev/null 127.0.0.1/catalog/25/women.aspx


#!/bin/bash

FILES=/var/log/nginx/wget/*

if [ -s /var/log/nginx/wgets.pid ] then
cat /var/log/nginx/wgets.pid | xargs kill
rm -f /var/log/nginx/wgets.pid
fi

do

cat $f | xargs wget & echo $! >> /var/log/nginx/wgets.pid

done

Now it remains only to update the cache pages when voting for a comment. This deals with the demon, constantly monitors the logs for a vote. 

#! / bin / bash 
# Get the authentication 
curl -H “Cookie: BasketID = KYKY-B-PYKY-U-ATAC” -H “Content-Type: application / json” -d “{\” login \ “: \ “USERNAME \”, \ “password \”: \ “PASS \”, \ “remeberMe \”: \ “true \”} ” -c /var/log/nginx/cookies.txt http: // 127.0 . 0.1/Resources/Services/SystemService.asmx/SignIn 

line=$(tail -n 1 /var/log/nginx/cookies.txt)
AUTH=$(echo $line | awk '{wild=$7; print wild}')

if [ -f /var/log/nginx/post-daemon.pid ] ;
then
echo "POST-daemon already running!"
exit
fi
(/usr/bin/tail -f /var/log/nginx/proxy_access.log & echo $! >/var/log/nginx/post-daemon.pid) |
while read -r line
do
if [[ $line =~ '/Resources/Services/SystemService.asmx/VoteToComment|POST|200' ]];
then
ref=$(echo $line | awk -F"|" '{ FS="|"; ref=$6; print ref}')
sortOrder=$(echo $line | awk -F"|" '{ FS="|"; co=$7; print co}')
IsAuth=$(echo $line | awk -F"|" '{ FS="|"; IsAuth=$8; print IsAuth}')
if [[ $sortOrder == "-" ]];
then
sortOrder=""
fi
if [[ $IsAuth == "0" ]];
then
wget --user-agent="WGET-POST-daemon" --header="Cookie: __sortOrder=$sortOrder" -o /dev/null -O /dev/null $ref
else
wget --user-agent="WGET-POST-daemon" --header="Cookie: __sortOrder=$sortOrder; AUTH=$AUTH" -o /dev/null -O /dev/null $ref
fi
fi
done
exit

Configuration for logrotate:


/var/log/nginx/*log {
daily
rotate 24
size 1
missingok
notifempty
compress

postrotate
/etc/init.d/nginx reload
/etc/init.d/nginx-POSTcache restart
endscript
}

init script for our daemon:

#!/bin/sh
#
# This script starts and stops the nginx cache updater daemon
#

# chkconfig:   - 85 15 
#
# processname: post-daemon
# pidfile: /var/log/nginx/post-daemon.pid

. /etc/rc.d/init.d/functions

daemon="/usr/local/sbin/post-daemon.sh"
pidfile="/var/log/nginx/post-daemon.pid"
prog=$(basename $daemon)

start() {
    [ -x $daemon ] || exit 5

    echo -n $"Starting POST-daemon: "
    ($daemon &) &
    retval=$?
    echo
    [ $retval -eq 0 ]

    return $retval
}

stop() {
    echo -n $"Stopping POST-daemon: "
    pid=$(cat $pidfile)
    kill $pid

    rm -f $pidfile
    retval=$?
    echo
    [ $retval -eq 0 ] && rm -f $lockfile

    return $retval
}

restart() {
    stop
    start
}

rh_status() {
    status $prog
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        restart
        ;;
    status)
        rh_status
        ;;
    *)
        echo $"Usage: $0 {start|stop|status|restart}"

        exit 2
esac

The solution is slightly crusty, but working. The load on IIS fell, the speed of return of pages to the client increased. I note that all the pictures of the site are on a separate server and can not be cached.