Rescue gentoo with software raid-1

Chroot to Gentoo env

livecd ~ # modprobe raid1
livecd ~ # mdadm --detail --scan
ARRAY /dev/md/3_0 metadata=0.90 UUID=5af462ee:8e535294:cb201669:f728008a
ARRAY /dev/md/2_0 metadata=0.90 UUID=09338879:65611484:cb201669:f728008a
ARRAY /dev/md/1_0 metadata=0.90 UUID=2fc18a05:fd33a57f:cb201669:f728008a
livecd ~ # mdadm --detail --scan > /etc/mdadm.conf
livecd ~ # ls /dev/md
md/       md125     md126     md127     mdev.seq  
livecd ~ # ls /dev/md
md/       md125     md126     md127     mdev.seq  
livecd ~ # ls /dev/md/
1_0  2_0  3_0  
livecd ~ # mount /dev/md/3_0 /mnt/gentoo/
livecd ~ # ls /mnt/gentoo/
bin  boot  dev  etc  home  lib  lib64  lost+found  media  mnt  opt  proc  root  run  sbin  sys  tmp  usr  var
livecd ~ # mount /dev/md/1_0 /mnt/gentoo/boot 
livecd ~ # mount -t proc proc /mnt/gentoo/proc
livecd ~ # mount --rbind /dev /mnt/gentoo/dev
livecd ~ # mount --rbind /sys /mnt/gentoo/sys
livecd ~ # cp -L /etc/resolv.conf /mnt/gentoo/etc/ 
livecd / # chroot /mnt/gentoo /bin/bash                      
livecd ~ # source /etc/profile

Do kernel stuff

cd /usr/src/linux
make && make modules_install
make install

Grub2 stuff: add additional parameters to kernel
nano -w /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT=”domdadm rootfstype=ext4″

Initframes

# emerge genkernel
# genkernel --install initramfs

To configure genkernel add your mdadm.conf file to your initrd with the “MDADM_CONFIG=” option in /etc/genkernel.conf

nano -w etc/genkernel.conf

MDADM_CONFIG="/etc/mdadm.conf"

If you need specific support in the initramfs, such as lvm or raid, add in the appropriate options to genkernel. See genkernel –help for more information, or the next example which enables support for LVM and software raid (mdadm):

# genkernel --lvm --mdadm --install initramfs

grub2 setup

grub2-mkconfig -o /boot/grub/grub.cfg

check result in /boot/grub/grub.cfg and compare raid device id with

blkid | grep /dev/md3

Reboot

livecd linux-3.15.10-hardened-r1 # exit
exit
livecd ~ # umount -l /mnt/gentoo/dev{/shm,/pts,}
livecd ~ # umount -l /mnt/gentoo{/proc,/boot,/sys,}
livecd ~ # reboot

Gentoo install min on USB stick

Download minimal CD ISO from gentoo.org


dmesg | tail
parted /dev/sdX
dd if=gentoo_image.img of=/dev/sdX bs=4M

Longer method

Taken from gentoo-en.vfose.ru
Install syslinux

emerge syslinux

Find the USB drive

fdisk -l

Clear disk partition table and boot sector

dd if=/dev/zero of=/dev/sdf count=1024

Create a partition table with 512-byte sectors and a primary partition of type 83 (ext2) with fdisk and make it bootable

fdisk -b 512 /dev/sdf
n p 1 enter enter
t 83
a 1 
p
w
Device     Boot      Start        End      Blocks   Id  System
/dev/sdf1   *           1        1009      500433   83  Linux

Format the partition ext2 and tune it

mkfs.ext2 /dev/sdf1
tune2fs -i 0 /dev/sdf1

install the mbr

equery f sys-boot/syslinux | grep mbr

Note: the listing above is for gentoo only. If you are using ubuntu, the listing above can be ignored.

cat /usr/share/syslinux/mbr.bin > /dev/sdf

If cat command returns Permission Denied error try with:

dd if=/usr/share/syslinux/mbr.bin of=/dev/sdf

mount the usb flash drive and disk image. If the device folder doesn’t exist, please create them before conducting the following listing.

mount -t ext2 /dev/sdf1 /mnt/usb/
mount -t iso9660 -o loop,user /downloads/install-x86-minimal-2008.0.iso /mnt/cdrom

install the system files

cd /mnt/usb/
cp -r /mnt/cdrom/* /mnt/usb/
mv isolinux extlinux
mv extlinux/isolinux.cfg extlinux/extlinux.conf
rm extlinux/isolinux.bin
rsync -av extlinux 

install the bootloader

cd /tmp
extlinux -i /mnt/usb/extlinux
umount /mnt/usb 

Lastly ensure the changes are synced to the USB Stick before removing it:

sync

Unbricking WRT54G

A quick guide how to ‘fix’ bricked WRT54G. I managed to brick it by installing OpenWRT firmware (attitude adjustment v12.09). Unfortunately this router doesn’t have enough memory to handle v12.09. My attempts to upload a new firmware trough ssh bricked the router.

To unbrick it I first did 30 30 30 reset.

  • Powered router hold reset button for 30 sec
  • Still holding reset, unpower router and wait 30 sec
  • Still holding reset power on the router and hold for another 30 sec

Find a working firmware and upload it using tftp. I used tomato WRT54G_WRT54GL.bin). Make sure tftp is installed on your system

emerge -av net-ftp/atftp
emerge -av net-ftp/linksys-tftp

Connect your computer to the router and set a static IP. On my linux box this is done by the following steps


/etc/init.d/net.eth1 stop
nano -w /etc/conf.d/net

routes_eth1="default via 192.168.1.1"
config_eth1="192.168.1.15 netmask 255.255.255.0 brd 192.168.1.255"

/etc/init.d/net.eth1 start

Turn on the router and check if the router can be pinged. Do a router power cycle and let ping command working
ping 192.168.1.1
If the router can be pinged most likely you will be able to upload a new firmware using tftp.

TFTP flashing instructions

Turn off the router. Initialize ftp transfer:

atftp 192.168.1.1
verbose
trace
timeout 60
put WRT54G_WRT54GL.bin

and turn on the router. The transfer should start by itself. If tfpt gets disconnected you can repeat the whole process.

atftp 192.168.1.1
tftp> timeout 60
tftp> trace
Trace mode on.
tftp> verbose
Verbose mode on.
tftp> put WRT54G_WRT54GL.bin
sent WRQ >
source port mismatch, check bypassedtimeout: retrying...
sent WRQ >
received ACK
sent DATA
received ACK
sent DATA
received ACK
sent DATA
received ACK

Another useful package is linksys-tftp

emerge -av net-ftp/linksys-tftp

To use it, first connect and then set the transfer


linksys-tftp> connect [router]
linksys-tftp> put [firmware image]

Useful resources

Bricked-Or-How-to-Resurrect-a-Dead-Linksys-WRT54G
openwrt wrt54g page

Net watchdog for OpenWrt

Lately, my cable connection gets disconnected too often. To automate reconnection, I installed a simple watchdog script that pings outside address and if there is no response it restarts router network. It’s not most efficient, since only WAN needs to be restarted, but it does the job.

Here is a simple watchdog script.

#!/bin/sh
n=0
while [ $n -lt 10 ]
do
  if /bin/ping -c 1 8.8.8.8 >/dev/null
  then
    exit 0
  fi
  n=$((n+1))
done
/etc/init.d/network restart

First, ssh to router and create this script

ssh root@192.168.2.1
nano /root/watchdog.sh
chmod +x /root/watchdog.sh

Save it to /root/watchdog.sh, and give executable permissions
chmod +x /root/watchdog.sh
Add it to cron in order to execute it every minute:

* * * * * /root/watchdog.sh

You can either add this line using luci web interface at
http://192.168.1.1/cgi-bin/luci/admin/system/crontab/
or just edit /etc/crontabs/root file.

Make sure cron is enabled!

Replacing failed disk in software RAID

mark the drive as failed first

mdadm /dev/mdX -f /dev/sdYX

Power down and physically replace faulty drive. Make sure the partitions are properly aligned, as in http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/#tools

Boot and partition new drive, and if you want to partition the new drive similar as others, you can do it with..

sfdisk -d /dev/sdX | sfdisk /dev/sdY
mdadm --detail /dev/mdX
mdadm /dev/mdX -a /dev/sdYX
watch -n .1 cat /proc/mdstat

In case failed disk has not been removed from the raid, we might need to assemble raid manually.
First, examine current raid configuration, and if needed update system mdadm.conf.

mdadm --examine --scan
nano -w /etc/mdadm.conf

Assemble raid:

mdadm --assemble /dev/mdX
mdadm --assemble /dev/mdX --scan
mdadm --assemble /dev/mdX --scan --force

In case if the assembling fails, try to stop raid and assemble it after

mdadm --stop /dev/mdX
mdadm --assemble /dev/mdX --scan --force

finally, add a new drive

mdadm --manage /dev/mdX --add /dev/sdYX

RAID repair
echo repair >> /sys/block/md1/md/sync_action
echo repair >> /sys/block/md2/md/sync_action
echo repair >> /sys/block/md3/md/sync_action

RAID maintenance and check
echo check >> /sys/block/md1/md/sync_action
echo check >> /sys/block/md2/md/sync_action
echo check >> /sys/block/md3/md/sync_action

Windows 7 boot fix

Win7 unable to boot. After BIOS, computer restarts. Seems like boot sector error. Linux utils unable to find anything wrong. Boot win7 installation disk and select REPAIR, go to command prompt and run following instructions.


bootrec.exe /fixboot
bootrec.exe /fixmbr

Quick winXP/Vista/7/8 recovery

To a friend in trouble.

In short winXP doesn’t behave nice. Most likely virus. Backup and full reinstall needed. Here is my advice how to back it up using linux live distro.

1. install linux live cd to CD or better usb key
2. boot old computer using linux from step 1 — DONT install linux, just use LIVE CD to boot computer, you don’t want to rewrite you hard drive!
3. access hard disk and copy files you need to other usb key/external disk/other computer
4. never again use windows – use linux from 1 to make it permanent on that PC
5. go to 4

In principle each step might open new problems, but hopefully all will go smooth.
In more detail I recommend kubuntu live cd:

1a. http://www.kubuntu.org/getkubuntu – most likely you need 32 bit version – for old comp,
1b. Here is the link to how to install it to USB stick.

2. when rebooting you might have to change boot medium in BIOS. Press key to get to BIOS when booting (before windows starts). Once in BIOS look for Boot priorities and adjust to use USB or CD to boot from (depending on step 1)

3. open file browser (dolphin, or which ever came with the linux distro) and look for primary hard disk partitions – it should be visible on left side. Mount it (click on it) and you will be able to browse windows partition(s). You can copy files to another USB stick the same way: insert other USB device (but not remove original linux LIVE CD medium), mount it and copy files to it.

The best guide I found on the net is
here.

Hope this helps,

Large HD reminder

Prepare hard drive with parted. It is recommended to ask parted to use optimal partition alignment:

parted -a optimal /dev/sdX

(parted) help
(parted) print
(parted) mklabel gpt
(parted) mkpart primary ext4 0% 100%
(parted) name 1 storage_01
(parted) print
(parted) quit

create filesystem
mkfs.ext4 /dev/sdX1

To remove hot-plugged SATA disk, first unmount then suspend disk

umount /dev/sdXn
hdparm -y /dev/sdX
hdparm -Y /dev/sdX

Repair ext4 partition

fsck.ext4 -cDft -C 0 /dev/sd**

-c – check for bad sectors
-D – optimize directories if possible
-f – force check, even if filesystem seems clean
-t – print timing stats (use -tt for more)
-y – assume answer “yes” to all questions (such as, “do you want to continue”)
-C 0 – print progress info to stdout

Hard drive tests

A simple write test that writes zeroes to the entire disk
dd if=/dev/zero of=/dev/sdX

3 TB drive results

dd: writing to ‘/dev/sdc’: No space left on device
5860533169+0 records in
5860533168+0 records out
3000592982016 bytes (3.0 TB) copied, 58234.7 s, 51.5 MB/s

hdparm test

hdparm -Tt /dev/sdX

/dev/sdX:

/dev/sdc:
 Timing cached reads:   25128 MB in  2.00 seconds = 12580.01 MB/sec                                                                                                                                      
 Timing buffered disk reads: 560 MB in  3.00 seconds = 186.56 MB/sec 

A few passes with badblocks should provide a good test of your new hard drive.

emerge -uav e2fsprogs
badblocks -nsv /dev/sdX

Never use the -w option on a device containing an existing file system. This option erases data! If you want to do write-mode testing on an existing file system, use the -n option instead. It is slower, but it will preserve your data.

stress testing new hard drive with stress linux

emerge -av stress bonnie++
stress --help
stress --cpu 3 --io 1 --vm 1 --vm-bytes 128M --hdd 1 --hdd-bytes 1024K --timeout 10s
bonnie++

specific stress test of a mounted ssd disk

bonnie++ -d /path/to/mounted/ssd -r your-system-ram-size-in-MB
# for 16GB RAM
bonnie++ -d /mnt/mounted-ssd-001 -r 16000

testing new hard drive with smartmontools

emerge -av smartmontools

(turn on monitoring)
smartctl -s on /dev/sdX
=== START OF ENABLE/DISABLE COMMANDS SECTION ===
SMART Enabled.

(view all information)
# smartctl --all

(perform long test)
# smartctl --test=long /dev/sdX

(check log after the test)
# smartctl --log=error /dev/sdX

start smartmon daemon and to start it at boot time, add it your runlevel:

/etc/init.d/smartd start

rc-update add smartd default

To ensure the changes are synced to the USB disk before removing it:

sync

Kubuntu on ASUS Zenbook Prime UX31A-DH5

After a few problems I can confirm ASUS Zenbook Prime UX31A-DH5 working with Kubuntu 13.10. Here are the steps I did:

Get kubuntu from http://www.kubuntu.org/getkubuntu/download
At the time of writing, version 13.10 was the most recent. It should work with BIOS Secure Boot option.

Copy iso image to usb device

dmesg

[67675.270470] sdf: sdf1 sdf2
[67675.335259] sd 8:0:0:0: [sdf] No Caching mode page found
[67675.335261] sd 8:0:0:0: [sdf] Assuming drive cache: write through
[67675.335263] sd 8:0:0:0: [sdf] Attached SCSI removable disk

dd if=kubuntu-13.10-desktop-amd64.iso of=/dev/sdf

Create a partition where ubuntu will be installed. In Windows 8 under Settings -> Control Panel -> Administrative Tools -> Computer Management -> Storage -> Disk Management –> Shrink Volume

BIOS update
Updated bios to 219 version. Make sure you have correct model, in my case its UX31A. Go to assus support site and get the BIOS file. Copy the BIOS file onto a USB stick. Reboot and press the esc button to enter the BIOS menu. Then select “Enter Setup”. From the Advanced tab, choose the Easy Flash option and then select the BIOS file to update.

Installing from a Live USB
Reboot than press and hold ESC for boot menu. I had trouble booting from one USB key and had to use another. Eventually the USB device got recognized and I got grub2 menu. After selecting kubuntu, the screen went black!1&(^$!@(*^& I connected external monitor and viola, ubuntu was there. See this bug.

Selected manual partitions, created 4GB swap and ~33GB / ext4 partition and proceeded with install.

Update
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-update

Black screen after the 1st boot
Kernel 3.11 has a bug with intel4000
Laptop screen black due to this bug. It is fixed in kernel-3.12, however this patch is not included with Kubuntu-13.10 :(. Kernel can be updated to v 3.12 as described here:


mkdir kernel\ v3.12.2-trusty && cd kernel\ v3.12.2-trusty

wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12.2-trusty/linux-image-3.12.2-031202-generic_3.12.2-031202.201311291538_amd64.deb--2013-11-30 18:47:15-- http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12.2-trusty/linux-image-3.12.2-031202-generic_3.12.2-031202.201311291538_amd64.deb

wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12.2-trusty/linux-headers-3.12.2-031202_3.12.2-031202.201311291538_all.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12.2-trusty/linux-headers-3.12.2-031202-generic_3.12.2-031202.201311291538_amd64.deb

sudo dpkg -i linux-*.deb
sudo update-grub
sudo reboot now

My Zenbook now properly boots with laptop screen working as it is supposed to!

Other tips
Some other tips for power saving optimization include modifying kernel parameters:

sudo nano -w /etc/default/grub

modify to include:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_osi='!Windows 2012' pcie_aspm=force drm.vblankoffdelay=1 i915.semaphores=1 nmi_watchdog=0"

and update grub with
sudo update-grub

SSD tweeks
Increase the life of your SSD by reducing number of OS writes to SSD
sudo nano -w /etc/fstab

and add discard,noatime,nodiratime params to your SSD partition as well as tmpfs. noatime disable (or significantly reduce) disk writes whenever a file is read. nodiratime gives you the same functionality for directories. discard filesystem option is for automatic/online TRIM.

UUID=0d67e753-acb6-46c4-acb4-1bd4f14fe92c /               ext4    discard,noatime,nodiratime,errors=remount-ro 0       1
tmpfs /tmp tmpfs defaults,discard,noatime,mode=1777 0 0

Make sure deadline is used as IO Scheduler


sudo cat /sys/block/sda/queue/scheduler
noop [deadline] cfq

Swap
Most systems with more than 2GB RAM rarely use swap space except for hibernate. Reduce usage of swap space in order to minimise number of writes to swap with


sudo echo 1 > /proc/sys/vm/swappiness

alternatively, this can be done by modifying sysctl

sudo nano -w /etc/sysctl.d/99-sysctl.conf

to look like

vm.swappiness=1
vm.vfs_cache_pressure=50

Resolution
Current Xorg-server uses a display resolution setting of 96dpi by default. UX31E Zenbooks have a resolution of ~138dpi. The default value of 96dpi makes fonts look really small. You can list the current setting with:

xdpyinfo|grep resolution

Override the default value with

xrandr --dpi 138/eDP1

To keep this permanent add xrand call to /etc/X11/xinit/xinitrc
sudo nano -w /etc/X11/xinit/xinitrc

Touchpad
Right and middle mouse clicks are not working. This needs specific xinput instruct. A quick recipe is given here. Create an executable script:

sudo nano -w /usr/local/bin/touchpad.sh
chmod +x /usr/local/bin/touchpad.sh

and add the following code

#!/bin/bash 
xinput set-prop "ETPS/2 Elantech Touchpad" "Synaptics ClickPad" 1 
xinput set-prop "ETPS/2 Elantech Touchpad" "Synaptics Soft Button Areas" 1956 0 1737 0 1304 1955 1737 0 
syndaemon -i 1.7 -d -t -K 

Lastly start this script
/usr/local/bin/touchpad.sh

To make it permanent add it to SystemSettings–>Startup and Shutdown–>AutoStart

Skype
Skype is not in official repos, to install it add ‘partner’ repository.

sudo add-apt-repository "deb http://archive.canonical.com/ $(lsb_release -sc) partner"
sudo apt-get update
sudo apt-get install skype

Grub2 on gentoo

Grub2 is finally declared as stable in gentoo portage. While optional, migration from grub:0 to grub2 is straightforward.

Installation of grub2 is done in four steps.

First inspect your existing grub.conf file and make sure naming is correct. Grub2 has strict naming requirements and wants kernel images to be named as kernel-${version} or vmlinuz-${version} and an initramfs must be named initramfs-${version}.img, initramfs-genkernel-${version}, initramfs-genkernel-${arch}-${version}, initrd-${version}.img, initrd.img-${version}, initrd-${version}.gz, or initrd-${version}. The kernel filenames must match a corresponding kernel that is available in /boot. See gentoo migration guide.

nano -w /boot/grub/grub.conf

default 0
timeout 5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title GRUB2 Chainload
root (hd0,0)
kernel /boot/grub/i386-pc/core.img
boot

title Gentoo Linux 3.10.7-r1
root (hd0,0)
kernel /boot/kernel-3.10.7-gentoo-r1 root=/dev/sda2

title Windoz7
map (hd0,0) (hd1,0)
map (hd1,0) (hd0,0)
rootnoverify (hd1,0)
makeactive
chainloader +1

GRUB2 Chainload section is added in order to allow testing of GRUB2 configuration before adopting it permanently.

Secondly, install grub2 files on your /boot partition.

# grub2-install --grub-setup=/bin/true /dev/sda
Installation finished. No error reported.

Thirdly, scan for the existing kernels.

To probe for Windows, grub needs os-prober and ntfs3g packages
emerge -av os-prober ntfs3g


# grub2-mkconfig -o /boot/grub/grub.cfg
Generating grub.cfg ...
Found linux image: /boot/kernel-3.10.7-gentoo-r1
No volume groups found
Found Windows 7 (loader) on /dev/sdb1
done

grub2-mkconfig will create /boot/grub/grub.cfg. This file is automatically generated using templates from /etc/grub.d and settings from /etc/default/grub. If you intend to use automatic configuration, /boot/grub/grub.cfg shouldn’t be manually modified since it will be overwritten next time you run grub2 config.
NOTE: grub.cfg can be also manually configured on your own risk.

Finally, test the configuration by rebooting and selecting GRUB2 Chainload. If everything works as supposed to install grub2 and remove the grub:0.

mount /boot
grub2-install /dev/sda
grub2-mkconfig -o /boot/grub/grub.cfg
emerge -Ca sys-boot/grub:0

Upgrading/modifying kernel

Whenever you wish to make changes to kernel grub2 has to be reconfigured. Add kernel image to /boot/ and run grub2 configuration.

grub2-mkconfig -o /boot/grub/grub.cfg

Chanloading iso image with custom kernel arguments is described at . It involves creating /etc/grub.d/ rule.

nano -w /etc/grub.d/50_isofile

#!/bin/sh
exec tail -n +3 $0

menuentry "GRUB4DOS" {
	linux /grub4dos-0.4.4/grub.exe --config-file=/menu.lst
}

menuentry "SYSRESCUECD" {
	loopback loop /sysrecuecd-x86-2.3.1.iso
	linux  (loop)/isolinux/rescue64 nomodeset vga=791 docache setkmap=fr isoloop=/sysrecuecd-x86-2.3.1.iso
	initrd (loop)/isolinux/initram.igz
}

menuentry "STG3-AMD64" {
	set cmdline="dokeymap looptype=squashfs loop=/image.squashfs cdroot"
	loopback loop /stg3-amd64-.iso
	linux  (loop)/isolinux/gentoo $cmdline root=/dev/ram0 init=/linuxrc initrd=gentoo.igz
	initrd (loop)/isolinux/gentoo.igz
}

Make this rule file executable.

Grub2 with raid-1 disk

Prepare kernel usual way

cd /usr/src/linux
make && make modules_install
make install

To boot from raid, grub2 need additional stuff: add additional parameters to kernel and pass mdadm support via initramfs
nano -w /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT=”domdadm rootfstype=ext4″

Initframes

# emerge genkernel

To configure genkernel add your mdadm.conf file to your initrd with the “MDADM_CONFIG=” option in /etc/genkernel.conf

nano -w etc/genkernel.conf

MDADM_CONFIG="/etc/mdadm.conf"

If you need specific support in the initramfs, such as lvm or raid, add in the appropriate options to genkernel. See genkernel –help for more information, or the next example which enables support for LVM and software raid (mdadm):

# genkernel --lvm --mdadm --install initramfs

grub2 setup
grub2-mkconfig -o /boot/grub/grub.cfg
check result in /boot/grub/grub.cfg and compare raid device id with

blkid | grep /dev/md3