The Debricking Script
Disclaimer: I haven't used the full capabilities of this script. My initial interest (use) of the script has been to initialise a completely blank disk drive:
debrick.sh /dev/sdd rootfs.img destroy
The script:
- Checks that some input parameters have been supplied otherwise shows a help screen.
- Checks that is running as 'root' and so has the necessary power to do what it needs to do.
- Checks that the mdadm package has been installed.
- Ensures the mount point /mnt/md0 is present and not in use.
- Checks/extracts the command line parameters
- If no disk has been specified it tries to find the one used by searching for a disk with the right sort of partition table on it and stops if it can't find one.
- Requests confirmation that it is about to use the right disk.
- If no image file was specified it tries to find it locally or will download it from the Western Digital website.
- It creates and compiles a simple 'swap' program (see below).
- Requests a final 'go ahead'.
- If 'destroy' has been selected it:
- Creates a new partition table on the disk.
- Formats partition 4 as ext4
- Checks again that it is OK to proceed.
- Uses madm to set up raid on partition 1.
- Formats partition 1 as ext3
- Uses madm to set up raid on partition 2.
- Copies the img file to partition 2.
- Briefly mounts the file system to
- Copy a boot script to /boot
- Enable ssh
- Runs the swap program on partitions 1 and 2.
The swap program
This is necessary because there is the not unreasonable assumption that this script is being run on an Intel/AMD x86 platform. Such processors are 'little-endian' in that numbers are stored least significant byte first. However the Duo's processor is 'big endian' so this can lead to problems if a disk drive is moved from a little-endian environment environment to a big endian environment.
In part this can be dealt with by tight specifications which lay down exactly how data is stored. Unfortunately with the raid arrays used here (Version 0.9 of the spec) the data in the Superblock is endian sensitive. (This was fixed in later versions of the spec). This means that the mdadm tool we used to set up the arrays will have the numbers in the superblock back to front as far as the software on the Duo is concerned. The swap program turns them the right way round for the Duo.
When compiling the program several warning messages are displayed. To fix this edit the script and add the two bits highlighted in red:
int main(int argc, char *argv[]) { int fd, i; unsigned long size; unsigned long long offset,lseek64(); char super[4096];
Notes
The disk uses a GPT partition table
mklabel gpt mkpart primary 528M 2576M mkpart primary 2576M 4624M mkpart primary 16M 528M mkpart primary 4624M -1M
Partition 1 holds the OS.
Partition 2 holds another copy of the OS.
Partition 3 is a swap partition
Partition 4 is the main data partition.
The command that sets up a particular raid:
mdadm --create $rootfsRaid --verbose --metadata=0.9 --raid-devices=2 --level=raid1 --run $diskRoot1 missing
Create
- /dev/md0 the device
- --verbose
- metadata=0.9 (Use SuperBlock version 0.9 which is 'endian' sensitive)
- raid-devices=2 (will use 2 drives?)
- level=raid1 (Raid 1 = mirroring)
- run (ignore discrepancies)
- /dev/sda1 missing (only one member is being given here)
[Go back]
The Script
Download debrick.sh or view it below.
#!/bin/bash
#
# The purpose of the script is to reinstall the operating system (debrick) on
# a harddrive that has been extracted from the housing of a WD MyBook Live.
#
#help screen
if [ $# = 1 -a "$1" = "--help" ]; then
echo "
standard use of script is:
sudo ./debricker.sh the script will find out what disk to use, it will not
touch the partition tables and therefore perserves data.
it will look what the newest version of the firmware is
via internet and then search for it in current folder or
subfolders. if none is found it will download one.
possible options are:
/dev/sd? path to the disk from mybook live. if not given, the script
will figure it out on its own.
/*/*.img path to the firmware that will be written to the disk. if
not given, the script will search for and then download it.
destroy script will rewrite the partition table of disk,
this will not perserve data, must match /dev/sd?.
example
sudo ./debricker.sh /dev/sda /firmwares/mine.img destroy
"
exit 1
fi
echo
#check that requirements are fullfilled
if [ "$(id -u)" != "0" ]; then
echo -e "this script must be run as root.\n"
exit 1
fi
if ! which mdadm > /dev/null; then
echo -e "this script requires the mdadm package.\n"
exit 1
fi
#making sure the mountpoint is available
rootfsMount=/mnt/md0
if [ -e ${rootfsMount} ]; then
if mountpoint -q ${rootfsMount}; then
echo "${rootfsMount} needs to be unmounted."
exit 1;
fi
fi
test -d "./mnt" || mkdir -p "/mnt"
test -d "$rootfsMount" || mkdir -p "$rootfsMount"
#making sure that there is no raid unit running
rootfsRaid=/dev/md0
if [ -e $rootfsRaid ]; then
echo -e "\n$rootfsRaid already exists! you need to stop and remove it.\n"
exit 1;
fi
#standard choices
disk=notset
image_img=notset
destroy=false
#handles the arguments and sets options
for (( arg=1; arg<=${#}; arg++ ))
do
case ${!arg} in
/dev/sd?) disk=${!arg};;
*.img) image_img=${!arg};;
"destroy") destroy=true;;
*) echo "unknown argument: ${!arg}"
exit 1
esac
done
echo "********************** DISK **********************"
echo
#figure out what disk to use
if [ $disk = "notset" ]; then
for x in {a..z}
do
#avoid a to literal matching in order to avoid incompability.
if [ -e /dev/sd${x} ]; then
diskTest=$(parted --script /dev/sd${x} print)
if [ ! -e /dev/sd${x}0 -a ! -e /dev/sd${x}5 ]; then
if [[ $diskTest = *WD??EARS* ]]; then
if [[ $diskTest = *??00GB* ]]; then
if [[ $diskTest = *3*B*B*5??MB*primary* ]]; then
if [[ $diskTest = *1*B*B*2???MB*ext3*primary*raid* ]]; then
if [[ $diskTest = *2*B*B*2???MB*ext3*primary*raid* ]]; then
if [[ $diskTest = *4*B*B*GB*ext4*primary* ]]; then
if [ $disk != notset ]; then
echo "multiple disk founds, you must enter the path manually:"
echo " sudo ./debricker.sh /dev/sd?"
exit 1;
fi
disk=/dev/sd${x}
fi; fi; fi; fi; fi; fi; fi; fi
done
if [ $disk == notset ]; then
echo "script could not find a matching sd unit connected to system."
exit 1
fi
else
if [ ! -e $disk ]; then
echo "$disk does not exist."
exit 1;
fi
fi
echo -e "script will use the following disk: \n"
parted --script $disk print
read -p "is this REALLY the disk you want? [y] " -n 1
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
echo -e "\nuser did not confirm, nothing was done.\n"
exit 1;
fi
echo
diskRoot1=${disk}1
diskRoot2=${disk}2
diskSwap=${disk}3
diskData=${disk}4
echo
echo "********************** IMAGE **********************"
echo
#the image was not given as parameter
if [ $image_img = notset ]; then
#find out what the latest version of firmware is
if ! which curl > /dev/null; then
echo -e "\nthis script requires the curl package, either install it or specify image file.\n"
exit 1
fi
wdc_homepage=""
wdc_latestfirmware=$(curl "${wdc_homepage}?type=AP1NC&fw=01.03.03" 2> /dev/null | awk ' {
if ( match($0, "upgrade file" ) != 0 ) {
split($0, http, "\"");
print http[2];
exit 1;
}
}')
latestversion_simple=$(echo $wdc_latestfirmware | cut -d'-' -f 2)
latestversion_pattern=$(echo $latestversion_simple | sed 's/../&*/g;s/:$//')
if [ "$latestversion_simple" == "" -o "$latestversion_pattern" == "" ]; then
echo -e "\ncould not fetch the latest version!\n"
exit 1;
fi
echo "checking: ${latestversion_simple}"
echo "searching: ./*/*${latestversion_pattern}.img"
image_img=$(find ./ -type f -name "*${latestversion_pattern}.img" -print; 2>/dev/null)
#get the latest firmware either from subdirs or internet
case `echo $image_img | wc -w` in
0) echo "searching: ./*/*${latestversion_pattern}.deb"
test -d "./firmware" || mkdir -p "./firmware"
image_deb=$(find ./ -type f -name "*${latestversion_pattern}.deb" -print; 2>/dev/null)
image_img="./firmware/rootfs_$latestversion_simple.img"
if ! [ `echo $image_deb | wc -w` == 1 ]; then
image_deb="./firmware/rootfs_$latestversion_simple.deb"
echo
echo "downloading: $image_deb"
read -p "confirm [y]: " -n 1
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
exit 1
fi
echo
curl $wdc_latestfirmware > $image_deb
if [ $? != 0 ]; then
echo -e "\ndownloading encountered problems.\n"
exit 1;
fi
fi
echo
echo "will extract: $image_deb"
read -p "confirm [y]: " -n 1
echo
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
exit 1
fi
echo "extracting: ./firmware/rootfs_${latestversion_simple}.img"
ar p $image_deb data.tar.lzma | unlzma | tar -x -C ./firmware
if [ $? != 0 ]; then
echo -e "\nextraction encountered problems.\n"
exit 1;
fi
mv ./firmware/CacheVolume/upgrade/rootfs.img ./firmware/rootfs_${latestversion_simple}.img
rm -rf ./firmware/CacheVolume;;
1) echo "found: $image_img";;
*) echo -e "\nmultiple image_img files was found."
exit 1
esac
else
if [ ! -e $image_img ]; then
echo "$image_img does not exist."
exit 1;
fi
fi
#construct the swap program
echo "\
#include <unistd.h>
#include <stdlib.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/mount.h>
#define MD_RESERVED_BYTES (64 * 1024)
#define MD_RESERVED_SECTORS (MD_RESERVED_BYTES / 512)
#define MD_NEW_SIZE_SECTORS(x) ((x & ~(MD_RESERVED_SECTORS - 1)) - MD_RESERVED_SECTORS)
main(int argc, char *argv[])
{
int fd, i;
unsigned long size;
unsigned long long offset;
char super[4096];
if (argc != 2) {
fprintf(stderr, \"Usage: swap_super device\\n\");
exit(1);
}
fd = open(argv[1], O_RDWR);
if (fd<0) {
perror(argv[1]);
exit(1);
}
if (ioctl(fd, BLKGETSIZE, &size)) {
perror(\"BLKGETSIZE\");
exit(1);
}
offset = MD_NEW_SIZE_SECTORS(size) * 512LL;
if (lseek64(fd, offset, 0) < 0LL) {
perror(\"lseek64\");
exit(1);
}
if (read(fd, super, 4096) != 4096) {
perror(\"read\");
exit(1);
}
for (i=0; i < 4096 ; i+=4) {
char t = super[i];
super[i] = super[i+3];
super[i+3] = t;
t=super[i+1];
super[i+1]=super[i+2];
super[i+2]=t;
}
/* swap the u64 events counters */
for (i=0; i<4; i++) {
/* events_hi and events_lo */
char t=super[32*4+7*4 +i];
super[32*4+7*4 +i] = super[32*4+8*4 +i];
super[32*4+8*4 +i] = t;
/* cp_events_hi and cp_events_lo */
t=super[32*4+9*4 +i];
super[32*4+9*4 +i] = super[32*4+10*4 +i];
super[32*4+10*4 +i] = t;
}
if (lseek64(fd, offset, 0) < 0LL) {
perror(\"lseek64\");
exit(1);
}
if (write(fd, super, 4096) != 4096) {
perror(\"write\");
exit(1);
}
exit(0);
}" >./swap.c
gcc swap.c -o swap
rm swap.c
echo
echo "********************** IMPLEMENTATION **********************"
echo -e "
everything is now prepared!
device: $disk
image_img: $image_img
destroy: $destroy\n"
read -p "this is the point of no return, continue? [y] " -n 1
echo
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
exit 1;
fi
echo
#rewrite the partition table
if [ $destroy = true ]; then
backgroundPattern="${backgroundPattern:-0}"
dd if=/dev/zero of=$diskRoot1 bs=1M count=32
dd if=/dev/zero of=$diskRoot2 bs=1M count=32
dd if=/dev/zero of=$diskSwap bs=1M count=32
dd if=/dev/zero of=$diskData bs=1M count=32
badblocks -swf -b 1048576 -t ${backgroundPattern} ${disk} 16 0
sync
sleep 2
parted $disk --align optimal <<EOP
mklabel gpt
mkpart primary 528M 2576M
mkpart primary 2576M 4624M
mkpart primary 16M 528M
mkpart primary 4624M -1M
set 1 raid on
set 2 raid on
set 3 raid on
set 4 raid on
quit
EOP
sync
sleep 1
#blocksize 65536 is required by the hardware, you won't be able to mount if different.
mkfs.ext4 -b 65536 -m 0 $diskData
echo
read -p "destroying was done, would you like to continue with installation? [y] " -n 1
echo -e
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
exit 1;
fi
fi
#write the image to the raid disk
echo
sync
mdadm --create $rootfsRaid --verbose --metadata=0.9 --raid-devices=2 --level=raid1 --run $diskRoot1 missing
mdadm --wait $rootfsRaid
sync
sleep 2
mkfs.ext3 -c -b 4096 $rootfsRaid
sync
sleep 2
mdadm $rootfsRaid --add --verbose $diskRoot2
echo
echo -n "synchronize raid... "
sleep 2
mdadm --wait $rootfsRaid
sync
echo -e "done\n"
echo "copying image to disk... "
dd if=$image_img of=$rootfsRaid
mount $rootfsRaid $rootfsMount
cp $rootfsMount/usr/local/share/bootmd0.scr $rootfsMount/boot/boot.scr
echo "enabled" > $rootfsMount/etc/nas/service_startup/ssh
sync
sync
sync
umount $rootfsMount
rmdir $rootfsMount
mdadm --stop $rootfsRaid
./swap $diskRoot1
./swap $diskRoot2
rm ./swap
echo
echo "all done! device should be debricked!"
echo