Partitioning of 3T hard drives

use parted with

parted /dev/sdX

Most disks on x86/amd64 are prepared using an msdos label, thus supporting partitions up to 2T in size. To create larger partitions (2 TB and more), one must use a gpt label (the GUID Partition Type). With parted, this can be accomplished with mklabel gpt:

(parted) help
(parted) print
(parted) mklabel gpt
(parted) mkpart primary ext4 0% 100%
(parted) print
(parted) quit

Prepare partition with

mkfs.ext4 /dev/sdX

Mount and enjoy.

WebApp Error: : [Errno 24] Too many open files

Most likely the number of file descriptors not enough.
Check for number of file descriptors with

ulimit -a
ulimit -n

Increase in /etc/security/limits.conf


username hard nofile 3072
username soft nofile 2048

Another things to do, if above solution doesn’t help. As suggested at slashdot, increase number of max open connections in sysctl.conf, kernel headers and security limits.

Params that configure max open connections.

At /etc/sysctl.conf add:

net.core.somaxconn=131072
fs.file-max=131072

and then:

sudo sysctl -p

at /usr/include/linux/limits.h

change:

NR_OPEN = 65536

at /etc/security/limits.conf

add:

* soft nofile 65535
* hard nofile 65535

Mounting USB devices in KDE

After updating KDE to 4.9.5 my system stopped auto mounting external USB devices. The fix was to change udisk policy in


/usr/share/polkit-1/actions/org.freedesktop.udisks.policy


<action id="org.freedesktop.udisks.filesystem-mount">
<description>Mount a device</description>
<message>Authentication is required to mount the device</message>
<defaults>
<allow_any>no</allow_any>
<allow_inactive>yes</allow_inactive>
<allow_active>yes</allow_active>
</defaults>
</action>

Linux command-line cheatsheet

Using help

apropos topic_name

Show list of manual pages related to topic

man topic_name

Show manual page for specified topic

info topic_name

Show info page for specified topic (sometimes easier to read than man)

whatis command_name

One-line summary for command

whereis command_name

Displays information where binary command is located; alternatively: “type command_name”

Continue reading “Linux command-line cheatsheet”

Apache configuration for pylons

Recently some of our pylons servers experienced mysql overflow. Pylons app is served via apache 2.2 using worker mpm on gentoo with kernel 3.2. It uses sqlalchemy to access mysql server. The pylons error is:

>> self.pool.connect(),
Module sqlalchemy.pool:210 in connect
>> return _ConnectionFairy(self).checkout()
Module sqlalchemy.pool:371 in __init__
>> rec = self._connection_record = pool._do_get()
Module sqlalchemy.pool:685 in _do_get
>> (self.size(), self.overflow(), self._timeout))
TimeoutError: QueuePool limit of size 5 overflow 5 reached, connection timed out, timeout 30

Simultaneously, RAM hit 100% capacity and swap kicked in. Looking into memory use on the machine, mysql took ~6%, apache ~1%, and everything else was eaten by pylons. My apps use beaker for caching of certain functions. In principle, turning off cache might help, however it would increase system load and slowed down server response.

The quick workaround was to modify number requests per child in apache controlled by MaxRequestsPerChild global. This basically allows faster recycling of processes and freeing

/etc/apache2/modules.d/00_mpm.conf was modified to reduce MaxRequestsPerChild:

<IfModule mpm_worker_module>
StartServers 16
MinSpareThreads 85
MaxSpareThreads 125
ThreadsPerChild 48
MaxClients 768
MaxRequestsPerChild 2000
</IfModule>

Brother MFC-7460DN on 64bit gentoo

Network printer Brother MFC-7460DN is not supported by standard cups drivers, but it does work on ubuntu by following instructions from manufacturer web site. Here is how this network printer can be properly configured on 64 bit gentoo (I am using kernel gentoo-sources-3.0.6)

Printer

Cups and other needed packages

Emerge cups and few other needed dependencies. Start cupsd.


emerge -av cups rpm a2ps tcsh
/etc/init.d/cupsd start
rc-update add cupsd default

Drivers

Get lpr and cupwrapper drivers for MFC-7460DN from Brother’s driver download page. The versions at the time of writing are:

cupswrapperMFC7460DN-2.0.4-2.i386
mfc7460dnlpr-2.1.0-1.i386

Install drivers and create a symbolic link to the filter file


rpm -ihv --nodeps mfc7460dnlpr-2.1.0-1.i386.rpm
rpm -ihv --nodeps cupswrapperMFC7460DN-2.0.4-2.i386.rpm
ln -s /usr/lib/cups/filter/brlpdwrapperMFC7460DN /usr/libexec/cups/filter/brlpdwrapperMFC7460DN

Continue reading “Brother MFC-7460DN on 64bit gentoo”

ATI Radeon HD6950 2GB on gentoo

ATI HD6950 is a great GPU which is supposed be full supported by opensource radeon drivers. After lot of pain with opensource driver, I have installed closed source ati-drivers and the card works well. During the process I have tried to follow number of ati/radeon/fglrx guides. My goal was to get the card working on two monitors using single screen.

For start tried xf86-video-ati drivers with kernel gentoo-sources-3.0.6. Initially it worked fine. Xorg did all the work by default and I didn’t have to mess with xorg.conf. I only adjusted primary and secondary monitor using kde-4.7.2 system settings. After a few minutes of testing – junk pixels started appearing around open windows and eventually garbage completely took over my two monitors.
Continue reading “ATI Radeon HD6950 2GB on gentoo”

Kate and cleaning Byte Order Mark

Once upon time BOM stole a few hours….

Over the years, Kate editor become my favorite editor and even more. With constant active development kate is more powerful than ever. Initially, I needed a light kde editor for my python projects: kate performed great. Over the time, I used kate more and more, and now it become my primary tool.

Recently, I noticed that somehow my session was set to add Byte Order Mark (BOM) to UTF-8 files. The byte order mark is a Unicode character used to signal the byte order of a text file. More importantly, it might confuse the python templates libraries such as mako, or babel. In kate, you can see if BOM is enabled for a specific file by looking at Tools->Add Byte Order Mark (BOM). As you can suspect, when one opens hundreds of files and accidentally attaches BOM at each one, it would be quite painful to clean BOM manually: uncheck Tools->Add Byte Order Mark (BOM). Here is the solution: find all files with BOM with find and awk


find . -type f -print0 | xargs -0r awk '
/^\xEF\xBB\xBF/ {print FILENAME}
{nextfile}'

And to clean BOM from all *.mako files


find . -type f -iname *.mako -exec sed 's/^\xEF\xBB\xBF//' -i.bak {} \; -exec rm {}.bak \;

and everyone lived happy ever after…

Awstats & virtual hosts

Gentoo has finaly moved away from webapp-config and simplified updating awstats. The example of apache config file setting awstats for apache virtual hosts runnig wsgi application is given bellow. Prerequisites are apache proxy for wsgi app, and awstats visible at www.some_domain.com/awstats.pl using authentication.

    <VirtualHost *:80>
        ServerName www.some_domain.com
        Serveralias some_domain.com
        ServerAdmin admin@some_domain.com

        ErrorLog /var/log/apache2/www.some_domain.com-error.log
        CustomLog /var/log/apache2/www.some_domain.com-access.log combined

        # awstats config
        Alias /awstats/classes "/usr/share/awstats/wwwroot/classes/"
        Alias /awstats/css "/usr/share/awstats/wwwroot/css/"
        Alias /awstats/icon "/usr/share/awstats/wwwroot/icon/"
        ScriptAlias /awstats/ "/usr/share/awstats/wwwroot/cgi-bin/"
        ScriptAlias /awstats "usr/share/awstats/wwwroot/cgi-bin/awstats.pl”
        ScriptAlias /awstats.pl "usr/share/awstats/wwwroot/cgi-bin/awstats.pl”

        <Directory "/usr/share/awstats/wwwroot">
                AllowOverride None
                Options None
                Order allow,deny
                Allow from all

                AuthType Basic
                AuthName "AWStats authenticated zone"
                AuthUserFile /etc/awstats/.htpasswd
                Require valid-user
        </Directory>
        <Directory "/usr/share/awstats/wwwroot/cgi-bin">
                SetHandler cgi-script
                Options +ExecCGI
        </Directory>

        ProxyPass /awstats !
        ProxyPass /awstats.pl !

        ProxyPass / http://localhost:5005/ retry=5
        ProxyPassReverse / http://localhost:5001/
        ProxyPreserveHost On
        <Proxy *>
            Order deny,allow
            Allow from all
        </Proxy>
    </VirtualHost>

Awstats config file (in /etc/awstats/) with geo-ip (emerge dev-perl/Geo-IP)

LogFile="/var/log/apache2/www.some_domain.com-access.log"

LogType=W
LogFormat=1
LogSeparator=" "
HostAliases="localhost 127.0.0.1 REGEX[myserver\.com$]"
DNSLookup=2
DirCgi="/cgi-bin"
DirIcons="/awstats/icon"
AllowToUpdateStatsFromBrowser=0
AllowFullYearView=2

LevelForFileTypesDetection=1
LevelForWormsDetection=2

SiteDomain="www.some_domain.com"
DirData="/home/some_user/awstats"

LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"

To test configuration run
/usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=www.some_domain.com -update

If you wish cron to handle update on every hour:
crontab -e -usome_user

0 * * * * cd /etc/awstats/ && /usr/share/awstats/wwwroot/cgi-bin/awstats.pl  -config=www.some_domain.com -update  >/dev/null 2>&1

Error with log file

Error: LogFile parameter is not defined in config/domain file
Setup ('www.dajstadas.com' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

obviously check LogFile and read permissions, however, this might fail if you call awstats update outside /etc/awstats directory. Retry with


cd /etc/awstats/ && /usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=www.some_domain.com -update

Dora saves the Snow Princess

Today Ania completed on her own her first video game. Wii version of Dora saves the Snow Princess doesn’t teach much, but helps kids to figure out the basics of using wii game controller. In order to complete a specific action in a game a certain controller motion is needed. The game itself is repetitive and most likely if you are older than 5 you’ll find it boring.