Egyre nagyobb igény lesz rá, hogy amennyire csak lehet tömörítsük, egyszerűsítsük a service szolgáltatásaink adatfolyamait. Amennyiben egy blog szolgáltatást üzemelünk, vagy más hírportált, ahol az adatokat, a tartalmat a nagy mennyiségű szöveg jelenti, el se hinnénk mekkora sávszélességet takaríthatunk meg, ha a kimenő (esetleg bejövő) adatmennyiséget letömörítjük.

Persze nehéz ezt számszerűsíteni, de php, javascript, html, txt oldalak esetén közel 90%-át takaríthatjuk meg sávszélességünknek. Ez persze valamelyest több processzor teljesítményt igényel, de valamit valamiért.

A következő Howto azt foglalja össze, hogyan lehet Apache serverhez telepíteni a mod_deflate (tömörítést elősegítő) modult. Azt, hogy kell beállítani, és végül, hogy információt is kapjunk mennyire működik ez a mi adattömörítő próbálkozásunk, bemutatja hogyan lehet statisztikát kinyerni az Apache logfilekből, az Awstats segítségével. Előre megnyugtatok mindenkit, hogy a művelet nem kifejezetten igényel szaktudást. A lényeg tudd mit miért csinálsz, és kövesd az útmutatást. Sok sikert!

Howtos – Activating content compression on Apache

Introduction

mod_deflate (http://httpd.apache.org/docs-2.0/mod/mod_deflate.html)
is an Apache module which compresses content sent from the web server to the client browser. The server uses a compression library called zlib, which should not be confused with the Linux gzip/gunzip applications. zlib-compressed content will be expanded automatically and transparently by all recent browsers, i.e. Internet Explorer 4+, Mozilla, Firefox, Konqueror, Safari, Opera, Netscape 4+, etc.

Regular text is very well suited for compression and most non-image content on the WWW is sent as text of some sort. (X)HTML, CSS, and Javascript all fall into this category. Text files can usually be compressed by 70%-80% leading to substantial bandwidth savings and
thus to faster loading pages in the browser. The effect is particularly noticeable for clients on a dial-up connection.

There is a downside or rather trade-off to using mod_deflate (http://httpd.apache.org/docs-2.0/mod/mod_deflate.html). Cpu-load and memory use on the server will increase slightly for each httpd-thread. Unless the server is very busy, the increase should be negligible, however.

Apache configuration changes

Thanks to a modular configuration setup for Apache of Clarkconnect, it is possible to change various settings without editing the main configuration file, /etc/httpd/conf/httpd.conf. The only thing one has to check is whether Apache is loading the necessary module, which it should be by default. Look for the line:

LoadModule deflate_module modules/mod_deflate.so

about a sixth of the way down from the top. If it has been commented out for some reason, simply remove the # from the front of the line.

New configuration file

Most of the new code goes into a new file, /etc/httpd/conf.d/deflate.conf. Make sure that the file has the same uid/gid and permissions as the other files in that Directory. Explanation below the listing.

---

#
# mod_deflate adds server-side compression of certain output "on the fly".
# All modern browsers are able to handle the decompression. 
#

AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css \
text/javascript application/x-javascript

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio

LogFormat '%v %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" \
mod_deflate: In:%{instream}n Out:%{outstream}n:%{ratio}npct.' deflate

CustomLog logs/deflate_log deflate

---

AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css \
text/javascript application/x-javascript

Tells Apache which types of files to compress. Server-side scripts such as PHP or ASP will be sent as text/html.

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio

Defines three variables for logging use. They are (in order): Bytes before compression, bytes after compression, compression ratio (outstream/instream).

LogFormat '%v %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" \
mod_deflate: In:%{instream}n Out:%{outstream}n:%{ratio}npct.' deflate

Defines the new logging format called deflate. It is very much like the regular combined format except for %v, which shows the virtual domain, and the final part which uses the three variables defined above. %v is not strictly necessary but can be useful if you have several domains and get confused looking through logfiles. Also, it is possible to log all Apache trafic to a single file, in which case it is quite useful to
have the domainname in front of the lines.

CustomLog logs/deflate_log deflate

Tells Apache to log all output (using the new format) to a file called /var/log/httpd/deflate_log unless other instructions are given.

Apache Access logs

The deflate.conf file is sufficient to activate content compression and a format for logging has also been defined. The next step is using that new format for the various domains hosted on the server,- the default domain as well as virtual domains.

Existing domains

Each domain has a configuration file in the /etc/httpd/conf.d/ directory. For the default domain it is called default.conf. The virtual domain configuration files are named after the domain, e.g. mydomain.com.vhost. In each of the files look for a line similar to this:

CustomLog /var/log/httpd/mydomain.com_access_log combined

It should be changed to:

CustomLog /var/log/httpd/mydomain.com_access_log deflate

Restarting Apache after all of the changes with the command service httpd restart will change the logging format.

New domains

Editing the *.vhost files is also the recommended practice for new domains. If an automatic change for all new domains is desired, it is necessary to edit /var/webconfig/classes/httpd.class.
Find the function AddHost and look for the line

$entry .= "\tCustomLog /var/log/httpd/" . $domain . "_access_log combined\n";

Change it to:

$entry .= "\tCustomLog /var/log/httpd/" . $domain . "_access_log deflate\n"; 

Now all new virtual domains will use the deflate logging format by default.

The new log files

An example of a line in the new logging format is shown below. In this case the content compression reduced the file from 6852 bytes to 3006 bytes, a reduction by 57%. That is actually a fairly bad ratio as you will see in the next section.
mydomain.com 193.26.xxx.xxx - - [02/Mar/2005:01:14:43 +0100] "GET /backend/syndicate.rdf HTTP/1.0" 200 3024 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.6) Gecko/20050223 Firefox/1.0.1" mod_deflate: In:6852 Out:3006:43pct.

Making awstats show the results

The awstats configuration files are stored in /etc/awstats/. In that directory there are two global files, conf.default and awstats.model.conf. Both of these should be edited. Find the section:

# Show file types chart.
# Context: Web, Streaming, Ftp
# Default: HB, Possible column codes: HBC
ShowFileTypesStats=HB

And change it to:

# Show file types chart.
# Context: Web, Streaming, Ftp
# Default: HB, Possible column codes: HBC
ShowFileTypesStats=HBC

This change will tell awstats to show the compression data in the report. However, awstats should also be told how the new log files are formatted and that requires editing a system file. The configuration files for the various domains in /etc/awstats/ are rewritten every time a new report is generated, so the program which writes the files must be edited.

Find the file /usr/sbin/awstatsreports and look for the line:

echo "LogFormat=$FORMAT" >> /etc/awstats/awstats.$DOMAIN.conf

and comment it out:

# echo "LogFormat=$FORMAT" >> /etc/awstats/awstats.$DOMAIN.conf

Add the following just beneath it:

echo "LogFormat=\"%virtualname %host %other %logname %time1 %methodurl %code \
%bytesd %refererquot %uaquot %other %gzipin %gzipout\"" >> \
/etc/awstats/awstats.$DOMAIN.conf

Done editing. The new awstats reports will have an expanded section on compressed content as seen below:

Compression On tells you how much data was compressed and the Result numbers are included in the Bandwidth column. In this example a total of 393.08 MB data was compressed by 85% to 56.77 MB. As the total Bandwidth usage was approximately 500 MB (not shown) about 40% has been saved.

Forrás: www.clarkconnect.com