Logo

My Book Live Duo: debrick.sh 

The Debricking Script

Disclaimer: I haven't used the full capabilities of this script. My initial interest (use) of the script has been to initialise a completely blank disk drive:

debrick.sh /dev/sdd rootfs.img destroy

The script:

  1. Checks that some input parameters have been supplied otherwise shows a help screen.
  2. Checks that is running as 'root' and so has the necessary power to do what it needs to do.
  3. Checks that the mdadm package has been installed.
  4. Ensures the mount point /mnt/md0 is present and not in use.
  5. Checks/extracts the command line parameters
  6. If no disk has been specified it tries to find the one used by searching for a disk with the right sort of partition table on it and stops if it can't find one.
  7. Requests confirmation that it is about to use the right disk.
  8. If no image file was specified it tries to find it locally or will download it from the Western Digital website.
  9. It creates and compiles a simple 'swap' program (see below).
  10. Requests a final 'go ahead'.
  11. If 'destroy' has been selected it:
    1. Creates a new partition table on the disk.
    2. Formats partition 4 as ext4
  12. Checks again that it is OK to proceed.
  13. Uses madm to set up raid on partition 1.
  14. Formats partition 1 as ext3
  15. Uses madm to set up raid on partition 2.
  16. Copies the img file to partition 2.
  17. Briefly mounts the file system to
    1. Copy a boot script to /boot
    2. Enable ssh
  18. Runs the swap program on partitions 1 and 2.

The swap program

This is necessary because there is the not unreasonable assumption that this script is being run on an Intel/AMD x86 platform. Such processors are 'little-endian' in that numbers are stored least significant byte first. However the Duo's processor is 'big endian' so this can lead to problems if a disk drive is moved from a little-endian environment environment to a big endian environment.

In part this can be dealt with by tight specifications which lay down exactly how data is stored. Unfortunately with the raid arrays used here (Version 0.9 of the spec) the data in the Superblock is endian sensitive. (This was fixed in later versions of the spec). This means that the mdadm tool we used to set up the arrays will have the numbers in the superblock back to front as far as the software on the Duo is concerned. The swap program turns them the right way round for the Duo.

When compiling the program several warning messages are displayed. To fix this edit the script and add the two bits highlighted in red:

int main(int argc, char *argv[])
{
    int fd, i;
    unsigned long size;
    unsigned long long offset,lseek64();
    char super[4096];

Notes

The disk uses a GPT partition table

mklabel gpt
mkpart primary 528M  2576M
mkpart primary 2576M 4624M
mkpart primary 16M 528M
mkpart primary 4624M -1M

Partition 1 holds the OS.
Partition 2 holds another copy of the OS.
Partition 3 is a swap partition
Partition 4 is the main data partition.

The command that sets up a particular raid:

mdadm --create $rootfsRaid --verbose --metadata=0.9 --raid-devices=2 --level=raid1 --run $diskRoot1 missing

Create

[Go back]

The Script

Download debrick.sh or view it below.

#!/bin/bash
#
# The purpose of the script is to reinstall the operating system (debrick) on 
# a harddrive that has been extracted from the housing of a WD MyBook Live.
#

#help screen
if  [ $# = 1 -a "$1" = "--help" ]; then
echo "
standard use of script is:
    sudo ./debricker.sh     the script will find out what disk to use, it will not
                            touch the partition tables and therefore perserves data.
                            it will look what the newest version of the firmware is
                            via internet and then search for it in current folder or
                            subfolders. if none is found it will download one.

possible options are:
    /dev/sd?                path to the disk from mybook live. if not given, the script 
                            will figure it out on its own.
    /*/*.img                path to the firmware that will be written to the disk. if 
                            not given, the script will search for and then download it.
    destroy                 script will rewrite the partition table of disk,
                            this will not perserve data, must match /dev/sd?.

example
    sudo ./debricker.sh /dev/sda /firmwares/mine.img destroy
"
exit 1
fi

echo

#check that requirements are fullfilled
if [ "$(id -u)" != "0" ]; then
    echo -e "this script must be run as root.\n"
    exit 1
fi
if ! which mdadm > /dev/null; then
    echo -e "this script requires the mdadm package.\n"
    exit 1
fi

#making sure the mountpoint is available
rootfsMount=/mnt/md0
if [ -e ${rootfsMount} ]; then
    if mountpoint -q ${rootfsMount}; then
        echo "${rootfsMount} needs to be unmounted."
        exit 1;
    fi
fi
test -d "./mnt" || mkdir -p "/mnt"
test -d "$rootfsMount" || mkdir -p "$rootfsMount"

#making sure that there is no raid unit running
rootfsRaid=/dev/md0
if [ -e $rootfsRaid ]; then
    echo -e "\n$rootfsRaid already exists! you need to stop and remove it.\n"
    exit 1;
fi

#standard choices
disk=notset
image_img=notset
destroy=false

#handles the arguments and sets options
for (( arg=1; arg<=${#}; arg++ ))
do
    case ${!arg} in
    /dev/sd?)    disk=${!arg};;
    *.img)        image_img=${!arg};;
    "destroy")    destroy=true;;
    *)             echo "unknown argument: ${!arg}" 
                exit 1
    esac
done

echo "********************** DISK           **********************"
echo

#figure out what disk to use
if [ $disk = "notset" ]; then
    for x in {a..z}
    do
        #avoid a to literal matching in order to avoid incompability.
        if [ -e /dev/sd${x} ];                                then
        diskTest=$(parted --script /dev/sd${x} print)
        if [ ! -e /dev/sd${x}0 -a ! -e /dev/sd${x}5 ];            then
        if [[ $diskTest = *WD??EARS* ]];                        then
        if [[ $diskTest = *??00GB* ]];                            then
        if [[ $diskTest = *3*B*B*5??MB*primary* ]];                then
        if [[ $diskTest = *1*B*B*2???MB*ext3*primary*raid* ]];    then
        if [[ $diskTest = *2*B*B*2???MB*ext3*primary*raid* ]];    then
        if [[ $diskTest = *4*B*B*GB*ext4*primary* ]];            then
            if [ $disk != notset ];                                then
                echo "multiple disk founds, you must enter the path manually:"
                echo "   sudo ./debricker.sh /dev/sd?"
                exit 1;
            fi
            disk=/dev/sd${x}
        fi; fi; fi;    fi;    fi;    fi; fi; fi
    done

    if [ $disk == notset ]; then
        echo "script could not find a matching sd unit connected to system."
        exit 1
    fi
else
    if [ ! -e $disk ]; then
        echo "$disk does not exist."
        exit 1;
    fi
fi

echo -e "script will use the following disk: \n"
parted --script $disk print
read -p "is this REALLY the disk you want? [y] " -n 1
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
    echo -e "\nuser did not confirm, nothing was done.\n"
    exit 1;
fi
echo

diskRoot1=${disk}1
diskRoot2=${disk}2
diskSwap=${disk}3
diskData=${disk}4

echo
echo "********************** IMAGE          **********************"
echo

#the image was not given as parameter
if [ $image_img = notset ]; then
    #find out what the latest version of firmware is
    if ! which curl > /dev/null; then
        echo -e "\nthis script requires the curl package, either install it or specify image file.\n"
        exit 1
    fi
    wdc_homepage=""
    wdc_latestfirmware=$(curl "${wdc_homepage}?type=AP1NC&fw=01.03.03" 2> /dev/null | awk ' {
        if ( match($0, "upgrade file" ) != 0 ) {
            split($0, http, "\"");
            print http[2];
            exit 1;
        }
    }')
    latestversion_simple=$(echo $wdc_latestfirmware | cut -d'-' -f 2)
    latestversion_pattern=$(echo $latestversion_simple | sed 's/../&*/g;s/:$//')
    if [ "$latestversion_simple" == "" -o "$latestversion_pattern" == "" ]; then
        echo -e "\ncould not fetch the latest version!\n"
        exit 1;
    fi
    echo "checking:     ${latestversion_simple}"
    echo "searching:    ./*/*${latestversion_pattern}.img"
    image_img=$(find ./ -type f -name "*${latestversion_pattern}.img" -print; 2>/dev/null)

    #get the latest firmware either from subdirs or internet
    case `echo $image_img | wc -w` in
    0)  echo "searching:    ./*/*${latestversion_pattern}.deb"
        test -d "./firmware" || mkdir -p "./firmware"
        image_deb=$(find ./ -type f -name "*${latestversion_pattern}.deb" -print; 2>/dev/null)
        image_img="./firmware/rootfs_$latestversion_simple.img"
        if ! [ `echo $image_deb | wc -w` == 1 ]; then
            image_deb="./firmware/rootfs_$latestversion_simple.deb"
            echo
            echo "downloading:  $image_deb"
            read -p "confirm [y]:  " -n 1
            if ! [[ $REPLY =~ ^[Yy]$ ]]; then
                exit 1
            fi
            echo
            curl $wdc_latestfirmware > $image_deb
            if [ $? != 0 ]; then
                echo -e "\ndownloading encountered problems.\n"
                exit 1;
            fi
        fi
        echo
        echo    "will extract: $image_deb"
        read -p "confirm [y]:  " -n 1
        echo
        if ! [[ $REPLY =~ ^[Yy]$ ]]; then
            exit 1
        fi
        echo "extracting:   ./firmware/rootfs_${latestversion_simple}.img"
        ar p $image_deb data.tar.lzma | unlzma | tar -x -C ./firmware
        if [ $? != 0 ]; then
            echo -e "\nextraction encountered problems.\n"
            exit 1;
        fi
        mv ./firmware/CacheVolume/upgrade/rootfs.img ./firmware/rootfs_${latestversion_simple}.img
        rm -rf ./firmware/CacheVolume;;
    1)  echo "found:        $image_img";;
    *)  echo -e "\nmultiple image_img files was found."
        exit 1
    esac
else
    if [ ! -e $image_img ]; then
        echo "$image_img does not exist."
        exit 1;
    fi
fi

#construct the swap program
echo "\
#include <unistd.h>
#include <stdlib.h>
#include <fcntl.h>
#include <stdio.h>
#include <sys/mount.h>

#define MD_RESERVED_BYTES      (64 * 1024)
#define MD_RESERVED_SECTORS    (MD_RESERVED_BYTES / 512)

#define MD_NEW_SIZE_SECTORS(x) ((x & ~(MD_RESERVED_SECTORS - 1)) - MD_RESERVED_SECTORS)

main(int argc, char *argv[])
{
    int fd, i;
    unsigned long size;
    unsigned long long offset;
    char super[4096];
    if (argc != 2) {
        fprintf(stderr, \"Usage: swap_super device\\n\");
        exit(1);
    }
    fd = open(argv[1], O_RDWR);
    if (fd<0) {
        perror(argv[1]);
        exit(1);
    }
    if (ioctl(fd, BLKGETSIZE, &size)) {
        perror(\"BLKGETSIZE\");
        exit(1);
    }
    offset = MD_NEW_SIZE_SECTORS(size) * 512LL;
    if (lseek64(fd, offset, 0) < 0LL) {
        perror(\"lseek64\");
        exit(1);
    }
    if (read(fd, super, 4096) != 4096) {
        perror(\"read\");
        exit(1);
    }

    for (i=0; i < 4096 ; i+=4) {
        char t = super[i];
        super[i] = super[i+3];
        super[i+3] = t;
        t=super[i+1];
        super[i+1]=super[i+2];
        super[i+2]=t;
    }
    /* swap the u64 events counters */
    for (i=0; i<4; i++) {
        /* events_hi and events_lo */
        char t=super[32*4+7*4 +i];
        super[32*4+7*4 +i] = super[32*4+8*4 +i];
        super[32*4+8*4 +i] = t;

        /* cp_events_hi and cp_events_lo */
        t=super[32*4+9*4 +i];
        super[32*4+9*4 +i] = super[32*4+10*4 +i];
        super[32*4+10*4 +i] = t;
    }

    if (lseek64(fd, offset, 0) < 0LL) {
        perror(\"lseek64\");
        exit(1);
    }
    if (write(fd, super, 4096) != 4096) {
        perror(\"write\");
        exit(1);
    }
    exit(0);

}" >./swap.c

gcc swap.c -o swap
rm swap.c

echo
echo "********************** IMPLEMENTATION **********************"

echo -e "
everything is now prepared!
device:       $disk
image_img:    $image_img
destroy:      $destroy\n"

read -p "this is the point of no return, continue? [y] " -n 1
echo
if ! [[ $REPLY =~ ^[Yy]$ ]]; then
    exit 1;
fi
echo

#rewrite the partition table
if [ $destroy = true ]; then
    backgroundPattern="${backgroundPattern:-0}"

    dd if=/dev/zero of=$diskRoot1 bs=1M count=32
    dd if=/dev/zero of=$diskRoot2 bs=1M count=32
    dd if=/dev/zero of=$diskSwap bs=1M count=32
    dd if=/dev/zero of=$diskData bs=1M count=32
    badblocks -swf -b 1048576 -t ${backgroundPattern} ${disk} 16 0

    sync
    sleep 2

    parted $disk --align optimal <<EOP
mklabel gpt
mkpart primary 528M  2576M
mkpart primary 2576M 4624M
mkpart primary 16M 528M
mkpart primary 4624M -1M
set 1 raid on
set 2 raid on
set 3 raid on
set 4 raid on
quit
EOP

    sync
    sleep 1

    #blocksize 65536 is required by the hardware, you won't be able to mount if different.
    mkfs.ext4 -b 65536 -m 0 $diskData

    echo
    read -p "destroying was done, would you like to continue with installation? [y] " -n 1
    echo -e
    if ! [[ $REPLY =~ ^[Yy]$ ]]; then
        exit 1;
    fi
fi

#write the image to the raid disk
echo
sync
mdadm --create $rootfsRaid --verbose --metadata=0.9 --raid-devices=2 --level=raid1 --run $diskRoot1 missing
mdadm --wait $rootfsRaid
sync
sleep 2
mkfs.ext3 -c -b 4096 $rootfsRaid
sync
sleep 2

mdadm $rootfsRaid --add --verbose $diskRoot2
echo
echo -n "synchronize raid... "
sleep 2
mdadm --wait $rootfsRaid
sync
echo -e "done\n"
echo "copying image to disk... "
dd if=$image_img of=$rootfsRaid
mount $rootfsRaid $rootfsMount
cp $rootfsMount/usr/local/share/bootmd0.scr $rootfsMount/boot/boot.scr
echo "enabled" > $rootfsMount/etc/nas/service_startup/ssh
sync
sync
sync
umount $rootfsMount
rmdir $rootfsMount
mdadm --stop $rootfsRaid
./swap $diskRoot1
./swap $diskRoot2
rm ./swap

echo
echo "all done! device should be debricked!"
echo

 


Any comments? email me. Added May 2020