Unbricking WRT54G

A quick guide how to ‘fix’ bricked WRT54G. I managed to brick it by installing OpenWRT firmware (attitude adjustment v12.09). Unfortunately this router doesn’t have enough memory to handle v12.09. My attempts to upload a new firmware trough ssh bricked the router.

To unbrick it I first did 30 30 30 reset.

  • Powered router hold reset button for 30 sec
  • Still holding reset, unpower router and wait 30 sec
  • Still holding reset power on the router and hold for another 30 sec

Find a working firmware and upload it using tftp. I used tomato WRT54G_WRT54GL.bin). Make sure tftp is installed on your system

emerge -av net-ftp/atftp
emerge -av net-ftp/linksys-tftp

Connect your computer to the router and set a static IP. On my linux box this is done by the following steps


/etc/init.d/net.eth1 stop
nano -w /etc/conf.d/net

routes_eth1="default via 192.168.1.1"
config_eth1="192.168.1.15 netmask 255.255.255.0 brd 192.168.1.255"

/etc/init.d/net.eth1 start

Turn on the router and check if the router can be pinged. Do a router power cycle and let ping command working
ping 192.168.1.1
If the router can be pinged most likely you will be able to upload a new firmware using tftp.

TFTP flashing instructions

Turn off the router. Initialize ftp transfer:

atftp 192.168.1.1
verbose
trace
timeout 60
put WRT54G_WRT54GL.bin

and turn on the router. The transfer should start by itself. If tfpt gets disconnected you can repeat the whole process.

atftp 192.168.1.1
tftp> timeout 60
tftp> trace
Trace mode on.
tftp> verbose
Verbose mode on.
tftp> put WRT54G_WRT54GL.bin
sent WRQ >
source port mismatch, check bypassedtimeout: retrying...
sent WRQ >
received ACK
sent DATA
received ACK
sent DATA
received ACK
sent DATA
received ACK

Another useful package is linksys-tftp

emerge -av net-ftp/linksys-tftp

To use it, first connect and then set the transfer


linksys-tftp> connect [router]
linksys-tftp> put [firmware image]

Useful resources

Bricked-Or-How-to-Resurrect-a-Dead-Linksys-WRT54G
openwrt wrt54g page

Net watchdog for OpenWrt

Lately, my cable connection gets disconnected too often. To automate reconnection, I installed a simple watchdog script that pings outside address and if there is no response it restarts router network. It’s not most efficient, since only WAN needs to be restarted, but it does the job.

Here is a simple watchdog script.

#!/bin/sh
n=0
while [ $n -lt 10 ]
do
  if /bin/ping -c 1 8.8.8.8 >/dev/null
  then
    exit 0
  fi
  n=$((n+1))
done
/etc/init.d/network restart

First, ssh to router and create this script

ssh root@192.168.2.1
nano /root/watchdog.sh
chmod +x /root/watchdog.sh

Save it to /root/watchdog.sh, and give executable permissions
chmod +x /root/watchdog.sh
Add it to cron in order to execute it every minute:

* * * * * /root/watchdog.sh

You can either add this line using luci web interface at
http://192.168.1.1/cgi-bin/luci/admin/system/crontab/
or just edit /etc/crontabs/root file.

Make sure cron is enabled!

Replacing failed disk in software RAID

mark the drive as failed first

mdadm /dev/mdX -f /dev/sdYX

Power down and physically replace faulty drive. Make sure the partitions are properly aligned, as in http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/#tools

Boot and partition new drive, and if you want to partition the new drive similar as others, you can do it with..

sfdisk -d /dev/sdX | sfdisk /dev/sdY
mdadm --detail /dev/mdX
mdadm /dev/mdX -a /dev/sdYX
watch -n .1 cat /proc/mdstat

In case failed disk has not been removed from the raid, we might need to assemble raid manually.
First, examine current raid configuration, and if needed update system mdadm.conf.

mdadm --examine --scan
nano -w /etc/mdadm.conf

Assemble raid:

mdadm --assemble /dev/mdX
mdadm --assemble /dev/mdX --scan
mdadm --assemble /dev/mdX --scan --force

In case if the assembling fails, try to stop raid and assemble it after

mdadm --stop /dev/mdX
mdadm --assemble /dev/mdX --scan --force

finally, add a new drive

mdadm --manage /dev/mdX --add /dev/sdYX

RAID repair
echo repair >> /sys/block/md1/md/sync_action
echo repair >> /sys/block/md2/md/sync_action
echo repair >> /sys/block/md3/md/sync_action

RAID maintenance and check
echo check >> /sys/block/md1/md/sync_action
echo check >> /sys/block/md2/md/sync_action
echo check >> /sys/block/md3/md/sync_action

Windows 7 boot fix

Win7 unable to boot. After BIOS, computer restarts. Seems like boot sector error. Linux utils unable to find anything wrong. Boot win7 installation disk and select REPAIR, go to command prompt and run following instructions.


bootrec.exe /fixboot
bootrec.exe /fixmbr

Large HD reminder

Prepare hard drive with parted. It is recommended to ask parted to use optimal partition alignment:

parted -a optimal /dev/sdX

(parted) help
(parted) print
(parted) mklabel gpt
(parted) mkpart primary ext4 0% 100%
(parted) name 1 storage_01
(parted) print
(parted) quit

create filesystem
mkfs.ext4 /dev/sdX1

To remove hot-plugged SATA disk, first unmount then suspend disk

umount /dev/sdXn
hdparm -y /dev/sdX
hdparm -Y /dev/sdX

Repair ext4 partition

fsck.ext4 -cDft -C 0 /dev/sd**

-c – check for bad sectors
-D – optimize directories if possible
-f – force check, even if filesystem seems clean
-t – print timing stats (use -tt for more)
-y – assume answer “yes” to all questions (such as, “do you want to continue”)
-C 0 – print progress info to stdout

Hard drive tests

A simple write test that writes zeroes to the entire disk
dd if=/dev/zero of=/dev/sdX

3 TB drive results

dd: writing to ‘/dev/sdc’: No space left on device
5860533169+0 records in
5860533168+0 records out
3000592982016 bytes (3.0 TB) copied, 58234.7 s, 51.5 MB/s

hdparm test

hdparm -Tt /dev/sdX

/dev/sdX:

/dev/sdc:
 Timing cached reads:   25128 MB in  2.00 seconds = 12580.01 MB/sec                                                                                                                                      
 Timing buffered disk reads: 560 MB in  3.00 seconds = 186.56 MB/sec 

A few passes with badblocks should provide a good test of your new hard drive.

emerge -uav e2fsprogs
badblocks -nsv /dev/sdX

Never use the -w option on a device containing an existing file system. This option erases data! If you want to do write-mode testing on an existing file system, use the -n option instead. It is slower, but it will preserve your data.

stress testing new hard drive with stress linux

emerge -av stress bonnie++
stress --help
stress --cpu 3 --io 1 --vm 1 --vm-bytes 128M --hdd 1 --hdd-bytes 1024K --timeout 10s
bonnie++

specific stress test of a mounted ssd disk

bonnie++ -d /path/to/mounted/ssd -r your-system-ram-size-in-MB
# for 16GB RAM
bonnie++ -d /mnt/mounted-ssd-001 -r 16000

testing new hard drive with smartmontools

emerge -av smartmontools

(turn on monitoring)
smartctl -s on /dev/sdX
=== START OF ENABLE/DISABLE COMMANDS SECTION ===
SMART Enabled.

(view all information)
# smartctl --all

(perform long test)
# smartctl --test=long /dev/sdX

(check log after the test)
# smartctl --log=error /dev/sdX

start smartmon daemon and to start it at boot time, add it your runlevel:

/etc/init.d/smartd start

rc-update add smartd default

To ensure the changes are synced to the USB disk before removing it:

sync

Grub2 on gentoo

Grub2 is finally declared as stable in gentoo portage. While optional, migration from grub:0 to grub2 is straightforward.

Installation of grub2 is done in four steps.

First inspect your existing grub.conf file and make sure naming is correct. Grub2 has strict naming requirements and wants kernel images to be named as kernel-${version} or vmlinuz-${version} and an initramfs must be named initramfs-${version}.img, initramfs-genkernel-${version}, initramfs-genkernel-${arch}-${version}, initrd-${version}.img, initrd.img-${version}, initrd-${version}.gz, or initrd-${version}. The kernel filenames must match a corresponding kernel that is available in /boot. See gentoo migration guide.

nano -w /boot/grub/grub.conf

default 0
timeout 5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title GRUB2 Chainload
root (hd0,0)
kernel /boot/grub/i386-pc/core.img
boot

title Gentoo Linux 3.10.7-r1
root (hd0,0)
kernel /boot/kernel-3.10.7-gentoo-r1 root=/dev/sda2

title Windoz7
map (hd0,0) (hd1,0)
map (hd1,0) (hd0,0)
rootnoverify (hd1,0)
makeactive
chainloader +1

GRUB2 Chainload section is added in order to allow testing of GRUB2 configuration before adopting it permanently.

Secondly, install grub2 files on your /boot partition.

# grub2-install --grub-setup=/bin/true /dev/sda
Installation finished. No error reported.

Thirdly, scan for the existing kernels.

To probe for Windows, grub needs os-prober and ntfs3g packages
emerge -av os-prober ntfs3g


# grub2-mkconfig -o /boot/grub/grub.cfg
Generating grub.cfg ...
Found linux image: /boot/kernel-3.10.7-gentoo-r1
No volume groups found
Found Windows 7 (loader) on /dev/sdb1
done

grub2-mkconfig will create /boot/grub/grub.cfg. This file is automatically generated using templates from /etc/grub.d and settings from /etc/default/grub. If you intend to use automatic configuration, /boot/grub/grub.cfg shouldn’t be manually modified since it will be overwritten next time you run grub2 config.
NOTE: grub.cfg can be also manually configured on your own risk.

Finally, test the configuration by rebooting and selecting GRUB2 Chainload. If everything works as supposed to install grub2 and remove the grub:0.

mount /boot
grub2-install /dev/sda
grub2-mkconfig -o /boot/grub/grub.cfg
emerge -Ca sys-boot/grub:0

Upgrading/modifying kernel

Whenever you wish to make changes to kernel grub2 has to be reconfigured. Add kernel image to /boot/ and run grub2 configuration.

grub2-mkconfig -o /boot/grub/grub.cfg

Chanloading iso image with custom kernel arguments is described at . It involves creating /etc/grub.d/ rule.

nano -w /etc/grub.d/50_isofile

#!/bin/sh
exec tail -n +3 $0

menuentry "GRUB4DOS" {
	linux /grub4dos-0.4.4/grub.exe --config-file=/menu.lst
}

menuentry "SYSRESCUECD" {
	loopback loop /sysrecuecd-x86-2.3.1.iso
	linux  (loop)/isolinux/rescue64 nomodeset vga=791 docache setkmap=fr isoloop=/sysrecuecd-x86-2.3.1.iso
	initrd (loop)/isolinux/initram.igz
}

menuentry "STG3-AMD64" {
	set cmdline="dokeymap looptype=squashfs loop=/image.squashfs cdroot"
	loopback loop /stg3-amd64-.iso
	linux  (loop)/isolinux/gentoo $cmdline root=/dev/ram0 init=/linuxrc initrd=gentoo.igz
	initrd (loop)/isolinux/gentoo.igz
}

Make this rule file executable.

Grub2 with raid-1 disk

Prepare kernel usual way

cd /usr/src/linux
make && make modules_install
make install

To boot from raid, grub2 need additional stuff: add additional parameters to kernel and pass mdadm support via initramfs
nano -w /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT=”domdadm rootfstype=ext4″

Initframes

# emerge genkernel

To configure genkernel add your mdadm.conf file to your initrd with the “MDADM_CONFIG=” option in /etc/genkernel.conf

nano -w etc/genkernel.conf

MDADM_CONFIG="/etc/mdadm.conf"

If you need specific support in the initramfs, such as lvm or raid, add in the appropriate options to genkernel. See genkernel –help for more information, or the next example which enables support for LVM and software raid (mdadm):

# genkernel --lvm --mdadm --install initramfs

grub2 setup
grub2-mkconfig -o /boot/grub/grub.cfg
check result in /boot/grub/grub.cfg and compare raid device id with

blkid | grep /dev/md3

Partitioning of 3T hard drives

use parted with

parted /dev/sdX

Most disks on x86/amd64 are prepared using an msdos label, thus supporting partitions up to 2T in size. To create larger partitions (2 TB and more), one must use a gpt label (the GUID Partition Type). With parted, this can be accomplished with mklabel gpt:

(parted) help
(parted) print
(parted) mklabel gpt
(parted) mkpart primary ext4 0% 100%
(parted) print
(parted) quit

Prepare partition with

mkfs.ext4 /dev/sdX

Mount and enjoy.

WebApp Error: : [Errno 24] Too many open files

Most likely the number of file descriptors not enough.
Check for number of file descriptors with

ulimit -a
ulimit -n

Increase in /etc/security/limits.conf


username hard nofile 3072
username soft nofile 2048

Another things to do, if above solution doesn’t help. As suggested at slashdot, increase number of max open connections in sysctl.conf, kernel headers and security limits.

Params that configure max open connections.

At /etc/sysctl.conf add:

net.core.somaxconn=131072
fs.file-max=131072

and then:

sudo sysctl -p

at /usr/include/linux/limits.h

change:

NR_OPEN = 65536

at /etc/security/limits.conf

add:

* soft nofile 65535
* hard nofile 65535

Mounting USB devices in KDE

After updating KDE to 4.9.5 my system stopped auto mounting external USB devices. The fix was to change udisk policy in


/usr/share/polkit-1/actions/org.freedesktop.udisks.policy


<action id="org.freedesktop.udisks.filesystem-mount">
<description>Mount a device</description>
<message>Authentication is required to mount the device</message>
<defaults>
<allow_any>no</allow_any>
<allow_inactive>yes</allow_inactive>
<allow_active>yes</allow_active>
</defaults>
</action>

Linux command-line cheatsheet

Using help

apropos topic_name

Show list of manual pages related to topic

man topic_name

Show manual page for specified topic

info topic_name

Show info page for specified topic (sometimes easier to read than man)

whatis command_name

One-line summary for command

whereis command_name

Displays information where binary command is located; alternatively: “type command_name”

Continue reading “Linux command-line cheatsheet”