Apache configuration for pylons

Recently some of our pylons servers experienced mysql overflow. Pylons app is served via apache 2.2 using worker mpm on gentoo with kernel 3.2. It uses sqlalchemy to access mysql server. The pylons error is:

>> self.pool.connect(),
Module sqlalchemy.pool:210 in connect
>> return _ConnectionFairy(self).checkout()
Module sqlalchemy.pool:371 in __init__
>> rec = self._connection_record = pool._do_get()
Module sqlalchemy.pool:685 in _do_get
>> (self.size(), self.overflow(), self._timeout))
TimeoutError: QueuePool limit of size 5 overflow 5 reached, connection timed out, timeout 30

Simultaneously, RAM hit 100% capacity and swap kicked in. Looking into memory use on the machine, mysql took ~6%, apache ~1%, and everything else was eaten by pylons. My apps use beaker for caching of certain functions. In principle, turning off cache might help, however it would increase system load and slowed down server response.

The quick workaround was to modify number requests per child in apache controlled by MaxRequestsPerChild global. This basically allows faster recycling of processes and freeing

/etc/apache2/modules.d/00_mpm.conf was modified to reduce MaxRequestsPerChild:

<IfModule mpm_worker_module>
StartServers 16
MinSpareThreads 85
MaxSpareThreads 125
ThreadsPerChild 48
MaxClients 768
MaxRequestsPerChild 2000
</IfModule>

Brother MFC-7460DN on 64bit gentoo

Network printer Brother MFC-7460DN is not supported by standard cups drivers, but it does work on ubuntu by following instructions from manufacturer web site. Here is how this network printer can be properly configured on 64 bit gentoo (I am using kernel gentoo-sources-3.0.6)

Printer

Cups and other needed packages

Emerge cups and few other needed dependencies. Start cupsd.


emerge -av cups rpm a2ps tcsh
/etc/init.d/cupsd start
rc-update add cupsd default

Drivers

Get lpr and cupwrapper drivers for MFC-7460DN from Brother’s driver download page. The versions at the time of writing are:

cupswrapperMFC7460DN-2.0.4-2.i386
mfc7460dnlpr-2.1.0-1.i386

Install drivers and create a symbolic link to the filter file


rpm -ihv --nodeps mfc7460dnlpr-2.1.0-1.i386.rpm
rpm -ihv --nodeps cupswrapperMFC7460DN-2.0.4-2.i386.rpm
ln -s /usr/lib/cups/filter/brlpdwrapperMFC7460DN /usr/libexec/cups/filter/brlpdwrapperMFC7460DN

Continue reading “Brother MFC-7460DN on 64bit gentoo”

Awstats & virtual hosts

Gentoo has finaly moved away from webapp-config and simplified updating awstats. The example of apache config file setting awstats for apache virtual hosts runnig wsgi application is given bellow. Prerequisites are apache proxy for wsgi app, and awstats visible at www.some_domain.com/awstats.pl using authentication.

    <VirtualHost *:80>
        ServerName www.some_domain.com
        Serveralias some_domain.com
        ServerAdmin admin@some_domain.com

        ErrorLog /var/log/apache2/www.some_domain.com-error.log
        CustomLog /var/log/apache2/www.some_domain.com-access.log combined

        # awstats config
        Alias /awstats/classes "/usr/share/awstats/wwwroot/classes/"
        Alias /awstats/css "/usr/share/awstats/wwwroot/css/"
        Alias /awstats/icon "/usr/share/awstats/wwwroot/icon/"
        ScriptAlias /awstats/ "/usr/share/awstats/wwwroot/cgi-bin/"
        ScriptAlias /awstats "usr/share/awstats/wwwroot/cgi-bin/awstats.pl”
        ScriptAlias /awstats.pl "usr/share/awstats/wwwroot/cgi-bin/awstats.pl”

        <Directory "/usr/share/awstats/wwwroot">
                AllowOverride None
                Options None
                Order allow,deny
                Allow from all

                AuthType Basic
                AuthName "AWStats authenticated zone"
                AuthUserFile /etc/awstats/.htpasswd
                Require valid-user
        </Directory>
        <Directory "/usr/share/awstats/wwwroot/cgi-bin">
                SetHandler cgi-script
                Options +ExecCGI
        </Directory>

        ProxyPass /awstats !
        ProxyPass /awstats.pl !

        ProxyPass / http://localhost:5005/ retry=5
        ProxyPassReverse / http://localhost:5001/
        ProxyPreserveHost On
        <Proxy *>
            Order deny,allow
            Allow from all
        </Proxy>
    </VirtualHost>

Awstats config file (in /etc/awstats/) with geo-ip (emerge dev-perl/Geo-IP)

LogFile="/var/log/apache2/www.some_domain.com-access.log"

LogType=W
LogFormat=1
LogSeparator=" "
HostAliases="localhost 127.0.0.1 REGEX[myserver\.com$]"
DNSLookup=2
DirCgi="/cgi-bin"
DirIcons="/awstats/icon"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2

LevelForFileTypesDetection=1
LevelForWormsDetection=2

SiteDomain="www.some_domain.com"
DirData="/home/some_user/awstats"

LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"

To test configuration run
/usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=www.some_domain.com -update

If you wish cron to handle update on every hour:
crontab -e -usome_user

0 * * * * cd /etc/awstats/ && /usr/share/awstats/wwwroot/cgi-bin/awstats.pl  -config=www.some_domain.com -update  >/dev/null 2>&1

Error with log file

Error: LogFile parameter is not defined in config/domain file
Setup ('www.dajstadas.com' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

obviously check LogFile and read permissions, however, this might fail if you call awstats update outside /etc/awstats directory. Retry with


cd /etc/awstats/ && /usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=www.some_domain.com -update

Socks5 proxy and wget

I’ve tried to set wget bash loop over socks5 proxy and was surprised to find out that wget does not have included proxy support. First start your proxy tunnel on port 8080 with

ssh -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -f -C -N -D *:8080 user@my.proxy.server &> tunnel.log

My firefox utilizes this tunnel perfectly – the proxy config page has the following entries:

  Manual proxy configuration:
    SOCKS Proxy  127.0.0.1  Port 8080
    "SOCKS v5"  checked

Initially I’ve tried using wget –execute to set http_proxy with

wget -e "http_proxy = 127.0.0.1:8080" "http://url_to_get"

however it didn’t work giving me the following error.

Connecting to 127.0.0.1:8080... connected.
Proxy request sent, awaiting response... No data received.
Retrying.

Wget with Tsocks

The solution was to install tsocks. For gentoo just do

emerge -av tsocks

and modify the config file. Note that gentoo keeps the confing information in /etc/socks/tsocks.conf, while ubuntu keeps it in /etc/tsocks.conf

nano -w /etc/socks/tsocks.conf

server = 127.0.0.1
server_type = 5
server_port = 8080

If you used proxy on earlier versions of wget, remove any proxy cmds from ~/.wgetrc

Finally start your wget trough socks5 proxy using tsocks

tsocks wget http://url_to_get

starting two tsocks wget at the same time caused my system to slow down and gave me the following error:

Connecting to www.ccc.ccccc|192.xxx.xxx.xxxx|:443... failed: 
Transport endpoint is not connected.

After killing one of the processes, I got back my cpu.

Do you wanna do more? Try Tor.