[TIP/Config] Basic nginx config to work as a proxy with apache+mod_pagespeed

[navjottomer]

Hi I am sharing my nginx config file here, I created this config to work nginx as a proxy with mod_pagespeed. Keep not this is working fine for me but it may not work for you
without some minor changes.
My server is a Ubuntu + Apache +Mysql +PHP +Nginx +Mod_pagespeed box.
It is not a best config in the world and I would really appreciate if someone give good suggestions.
So Lets Start:

nginx.conf

Code: [Select]

user server-user server-user;
worker_processes 3;
pid /run/nginx.pid;

events {
worker_connections 768;
# multi_accept on;
}

http {

##
# Basic Settings
##
   
           
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;

# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
    server_tokens on;
include /etc/nginx/mime.types;
default_type application/octet-stream;

##
# Logging Settings
##

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

##
# Gzip Settings
##

gzip on;
gzip_disable "msie6";
#     gzip_comp_level 9;

gzip_vary on;
gzip_proxied any;
   gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

##
# nginx-naxsi config
##
# Uncomment it if you installed nginx-naxsi
##

#include /etc/nginx/naxsi_core.rules;

##
# nginx-passenger config
##
# Uncomment it if you installed nginx-passenger
##

#passenger_root /usr;
#passenger_ruby /usr/bin/ruby;
    include proxy_params;
    proxy_cache_path /var/nginx_cache levels=1:2
                     keys_zone=main:60m
                     max_size=1g inactive=60m;
    proxy_cache_path /var/run levels=1:2
                     keys_zone=main1:10m
                     max_size=512m inactive=10m;
    proxy_temp_path /var/nginx_temp;

        ##
# Virtual Host Configs
##
   
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;

#Performance Tuning
access_log off;
client_body_buffer_size 12K;
        client_header_buffer_size 2k;
        client_max_body_size 2m;
        large_client_header_buffers 4 2k;
}

osclass Virtual host config

Code: [Select]

server {
       
        listen   80; ## listen for ipv4; this line is default and implied
        #listen   [::]:80 default ipv6only=on; ## listen for ipv6
        root /your/osclass/path/;
        server_name yourdomain.com ; #change the server name to reflect yours
       
       
        # purging cache via individual url...
    location ~ /purge(/.*) {
       
    proxy_cache_purge main1 $scheme$host$1;
   
}

# pass requests for dynamic content to site

      location / {

        # If non GET/HEAD, don't cache & mark user as uncacheable  via cookie
        # POST requests and urls with a query string should always go to PHP
if ($request_method = POST) {
set $osclass "1";
add_header Set-Cookie "_posc=180; Domain=tuffclassified.com; Max-Age=180; Path=/";           
                        add_header X-Microcachable "0";
} 

      #if user logged in bypass cache 
      if ($http_cookie ~* "3n3ke") {
            set $osclass "1";
        }

        # Bypass cache if no-cache cookie is set
        if ($http_cookie ~* "_posc") {
            set $osclass "1";
        }
        # Setup var defaults
   proxy_buffering on;
           proxy_redirect off;
           set $cache_key $scheme$host$uri$is_args$args;
           proxy_cache main1;
           proxy_cache_key $cache_key;
           proxy_cache_valid 200 302 5s;
           
           proxy_cache_use_stale updating error timeout;
           
           proxy_cache_bypass $osclass;
           
           proxy_ignore_headers Cache-Control Expires Set-Cookie;
           proxy_hide_header Pragma;
           proxy_pass_header Set-Cookie;
           add_header X-Cache $upstream_cache_status;
           proxy_pass http://backend;
           
         }
         
         
         location ~* ^/(oc-admin|item/new|user/)
    {
       
        proxy_buffering off;
    proxy_redirect off;
    proxy_cache_bypass 1;
         
        proxy_pass http://backend;
    }

       
        # serve static files
       
location ~* ^.+.(jpg|jpeg|gif|png|ico|bz2|doc|xls|pdf|ppt|txt|tar|wav|bmp|rtf|mp3|avi|mov)$ {
        access_log off; log_not_found off; expires max; error_page 404 @fallback;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
 
        }
       
    location @fallback {
            access_log off;
            set $cache_key $scheme$host$uri$is_args$args;
          proxy_cache main;
          proxy_cache_key $cache_key;
          proxy_cache_valid 1d;
          proxy_pass http://backend;
          expires max;
          add_header X-Cache $upstream_cache_status;
         
              }
    # pass requests for js css minified by pagespeed and cache them
         location ~*^.+.(js|css)$ {
            access_log off;
          set $cache_key $scheme$host$uri$is_args$args;
          proxy_cache main;
          proxy_cache_key $cache_key;
          proxy_cache_valid 1d;
          proxy_pass http://backend;
          expires max;
          add_header X-Cache $upstream_cache_status;
                   
         }   
   
        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
         location ~ /\.ht {
                deny all;
        }
}
"3n3ke" is a name of cookie which is unique for every osclass installation.
Feel free to ask if you need help or want to share you suggestions .

[navjottomer]

also sharing my apachebench output:

$$$$$$$:~# ab -n 1000 -c 100 http://tuffclassified.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking tuffclassified.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        nginx
Server Hostname:        tuffclassified.com
Server Port:            80

Document Path:          /
Document Length:        10068 bytes

Concurrency Level:      100
Time taken for tests:   0.320 seconds
Complete requests:      1000
Failed requests:        0
   (Connect: 0, Receive: 0, Length: 0, Exceptions: 0)
Write errors:           0
Total transferred:      10395803 bytes
HTML transferred:       10102641 bytes
Requests per second:    3125.20 [#/sec] (mean)
Time per request:       31.998 [ms] (mean)
Time per request:       0.320 [ms] (mean, across all concurrent requests)
Transfer rate:          31727.46 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    8   2.5      8      13
Processing:     2   18  10.0     18     309
Waiting:        2   10  10.1     10     308
Total:          7   26  10.1     26     320

Percentage of the requests served within a certain time (ms)
  50%     26
  66%     27
  75%     28
  80%     28
  90%     30
  95%     31
  98%     32
  99%     33
 100%    320 (longest request)

[teseo]

Some suggestions for your nginx.conf:

Code: [Select]

worker_processes 3;
Put here the number of CPUs or CPU cores your machine has.

Your OS may schedule all workers on single CPU/core this can be avoided using worker_cpu_affinity.

Add this line below (this example is for 8 cores, leave only the four first values in case of 4 cores, and so on)

Code: [Select]

worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;

Code: [Select]

gzip_comp_level 5;
Reference: http://phrogz.net/effects-of-gzip_comp_level-on-html

Code: [Select]

gzip_min_length  1000;
Beter avoid compressing files too small, it's time consuming and the final file might as well be bigger than original.   


Security recommendations

Code: [Select]

server_tokens off;

Whether to send the Nginx version number in error pages and Server header.

Regards

[navjottomer]

Hi,
This is a 3 core cloud server that's why i set worker_process to 3. Though, anything above 2-4 workers is overkill as nginx will hit other bottlenecks before the CPU becomes an issue and usually you’ll just have idle processes.

Setting CPU affinity is not a good idea because i think Ubuntu OS cpu scheduler is far better than nginx. It may be good in some case where you want to run only nginx on server.(It can be dangerous if set improperly.)

Because all files are cached so setting gzip_comp_level to 5 is not going to give any comparable cpu time improvement. (Although, setting it to higher will save your bandwidth with high traffic sites. Although default is good). But this value is still debatable.

What are your suggestions to improve cacheability of Osclass? It would be nice to hear your suggestions in that.

Anyway thanks for sharing your ideas.
Regards
Navjot Tomer

[Legion]

i am realy confuse what is this?

[VineyPapie]

It's time consuming and the final file.

[gnoe]

Hi Navjot,
still operating nginx as a proxy for apache or moved to php-frm?

I'll configure a new box next month and will use nginx as a proxy and leave apache for php.
Which one of this two setups is better?

[navjottomer]

Hi Navjot,
still operating nginx as a proxy for apache or moved to php-frm?

I'll configure a new box next month and will use nginx as a proxy and leave apache for php.
Which one of this two setups is better?

I removed apache, now my full setup is with nginx+php5-fpm+pagespeed+APC. It create enough head-room to run web-server efficiently with minimum configuration hassle.