Tuesday, August 29, 2006

Hard Disk Backup and Restore using PBA

Hard Disk Backup and Restore Walk-through using the Personal Backup Appliance

[ There's a newer version of this guide ]

Download PBA at: http://pba-vm.sourceforge.net

Discuss PBA, Get Help, and Share Ideas on the Discussion Forums


This is a guide to using the Personal Backup Appliance to backup and restore your hard disk. The Personal Backup Appliance is an open source alternative to Ghost server. The Personal Backup Appliance, or PBA for short, is distributed as a VMware Virtual Appliance. To get started, go to the link above, download the latest release, unzip it, and power it on using VMware Player, a free download from http://www.vmware.com/download/player/
When you first power on the VM, Ubuntu Linux will boot up and automatically launch Firefox with the main information page.
At this point, everything is setup and ready to go on the server-end. The next step is to set things up on the client-end. Please take note of the IP address that's displayed on your VM. It will most likely differ from the IP address in the screenshot. Please also take note of your VM's url.

From another system on the network, go to the PBA VM's web page, download the PBA client CD iso image from the VM, and burn it onto a CD.
Once you have the client CD, boot it on a computer or VM on your network. To boot from CD, you may need to change the boot order in your BIOS to boot from cdrom before the hard disk. After the client CD boots, you will be greeted with a prompt asking for the IP address of your Personal Backup Appliance.
Type the IP address and press [ENTER].
Next, you will presented with a simple menu. Use the arrow keys to navigate and Press Enter to select. Select Backup.
At the Backup Filename: prompt, enter in a POSIX compliant filename. If it isn't compliant, just try again.

When backing up, you will either see a partition image, ntfsclone, or dd progress screen.

When the Backup operation is complete, you will see this screen.
If you select the Restore option from the main menu, you will be presented with a menu to choose the archive that you want to restore.
Choose the archive that you want to restore.

During the restore operation, you will see either see a partition image, ntfsclone, or dd progress screen.

When the Restore operation is completed, you will see this screen.
Select Quit and power off the system.

32 comments:

anm said...

Hi,
I haven't tried it as yet, but there are a couple of questions.
1. is it necessary to have it running in a VM?
2. Does it work for all linux/ windows?
3. Suppose I take a backup with this application - and my client machine crashed later. I want to get back the complete installation of my machine. Can I restore and just boot off the machine (assuming all hardware etc are same).

regards

anm said...

Sorry for the double post earlier. It gave cookie disabled errors when I tried to comment from firefox - but seems the comments went through just fine.
Another quick question - suppose I create a dedicated machine for backups(I would still like the backup appliance to run directly on the backup machine - without a VM) - does/ can the transfer of data from client happen securely/ encrypted? Thanks!

rxhui said...

There's no fundamental reason why the backup service needs to run in an VM. The reason it's packaged as a VM is to remove the complexity of having to setup a server so that even a novice user would not be intimidated. At present, there's no install script for installation to a real machine, though I'm considering this for the future either as a release or a howto guide.

In theory, barring scripting errors, the Personal Backup Appliance should support linux, windows, and any x86-based operating system, any partition layout.

The Personal Backup Appliance backup scheme can do a bare-metal restore. In other words, your hard disk can completely die, and you should be able to restore to a replacement hard disk.

At present, transfers using ntfsclone and dd are encrypted during transfer using ssh. Transfers using partimage are not encrypted. PBA uses ntfsclone, partimage, or dd depending on the partition type.

anm said...

Problem - I do not have a DHCP in my network. So I need a fixed IP for my device.
I am using windows xp as host OS.
I see VMNet2 and VMNet8 in my network connections.
I changed the properties of eth0 to 10.1.0.54 while my hosts IP is 10.1.0.2 subnet being 255.255.0.0 on all.
My gateway is 10.1.0.1 - I have set it on both guest and host.
From my host, I am able to open the page http://10.1.0.54 - however I am not able to even ping 10.1.0.54 from 10.1.0.1.

I would like to backup the image on 10.1.0.1 so it is important that this gets connected.

regards

rxhui said...

This may or may not help, but I found this guide to set up networking manually. You would need to do this after booting from the client CD on the machine that you want to back up.

http://www.fdlinux.com/networksetuphowto.html

anm said...

Thanks RXHUI for posting that tutorial. I got it working.

Ok - So finally I got my networking working. 10.1.0.1 has IPCop with BOT - so that blocked even 10.1.0.1 from pinging machines inside green network. So normally, bridged networking works out of box, one needs to be careful of the firewalls (I did disable my windows firewall too - but I relied on pinging from 10.1.0.1 to make sure outerworld is reachable - only if I tried with another machine on green network - I wouldn't have wasted so much of time)

Here is my experience with PBA

Success: I needed a backup of my upcoming IPCop installation - so that it can be restored by not so knowledgeable folks in a small timeframe. So I plugged in the client CD, exited to the shell, ran netcardconfig to setup fixed IP. Started pba.sh again and the image was backed up to the server pretty fast.
Restoring gave me a lot of nightmares. It kept on playing with my hdd for a long time. After an hour or so, we decided it is a failure, let's hard boot the machine. The machine did not freeze, I could move to another terminal and do a "top" to see what processes are working. There was simply no indication of what restore is doing for so long.
Then we booted the machine after restart and seemingly linux/ IPCop came up ok (except for the missing LAN cards it was expecting). So we took this hdd to the original machine and IPCop again came up nicely. This time the network card etc worked like charm. Success!
RXHUI - Is there a way of predicting that hard disk restore is complete? How much hours should one wait to ensure restore has happened successfully?

Failure: We then tried doing a backup of a winxp. We had a non registered/ activated freshly installed xp -fat32. Backup once again worked like a charm and it finished in reasonable time. Then we tried restoring it - and it failed - "error partitioning".
RXHUI - any clues? Could it be because my hard disk earlier had linux on it? Do I need some kind of cleanup on my harddisk before pba can partition it for windows?

I yet need to try windows with ntfs. I may run into licensing key issues. I need to roll out workstations with windows xp - however I need to register with different keys etc for each. Any help on this front is appreciated.

anm said...

Tried FAT32, NTFS - no luck. Initially it complained about disk space and VM used to crash. Then I cleaned up by removing all old images by rm * on /home/vmware/data/backup (edited images.toc to avoid non existing backups from showing up)
Many times it did show 100% complete, but .part and .mbr files were missing.
Need help! Where are you rxhui :)

rxhui said...

I'm not sure what's wrong. I have not seen the problems that you described before. You should not need to do anything special to windows partitions (fat or ntfs) before archiving other than making sure the filesystem isn't corrupted.)

What's your configuration?

System: physical or VM
Storage: IDE or SCSI or SATA

On the system that you are trying to restore, can you post the output of 'sfdisk -l'

If you forcefully quit out of the process, the .part and .mbr files will not be transferred to the server.

Another thing to check is the amount of free space on the system hosting the VM.

Also, your firewall situation on your network needs to allow access to ssh,http, and port 4025 on the PBA VM.

anm said...

Thanks rxhui for quick response. I am not close to the systems today. Question - does the size of virtual data disk matter? What is the capacity of the virtual disk that came with PBA? Can it be increased to say something like 20GB? PBA vm runs from d drive of a machine and it has about 25GB free.
I will try further on monday. Tillt that time, if you can provide a larger vm disk, it would be great :)

rxhui said...

From the zip package that you downloaded, take a look in the hdd_upgrade folder. There's a readme.txt file that describes how to upgrade the capacity of your data vmdk drive. By default, the data vmdk disk is 20GB, but you can upgrade it up to 250GB, provided your host drive actually has that much space free.

anm said...

News: I installed backed up copy of my IPCop on a disk. Read earlier post about success. This morning I was not able to ping my internet router with it. I shut it down from the GUI. It would not reboot with the error /dev/harddisk4: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
(i.e., without -a or -p options)
Checking root filesystem: Failed

I tried to restore it again from the backup but this error would not go. I tried running fsck manually too but no luck. I am trying it with another hdd.

regards

anm said...

Hi rxhui, another quick question - why does it always hang while restoring linux? In my case I backed up ipcop test installation sometime back - the first time I tried to restore it, I waited for hours, then decided to shutdown. It was this restored disk that gave me a problem today. I tried restoring it with the old backup, but again it did not show me the prompt even after saying 100% complete.

rxhui said...

I have never observed what you are describing. Hanging while restoring Linux is not what's supposed to happen. I have backed up and restored Linux many times without incident.

Also, if the restore doesn't complete, the contents of the disk cannot be relied upon. In other words, the disk will likely be partially written and the filesystem will likely be corrupt.

What filesystem is IPCop?
Is it ext3, reiserfs, or something else?

What does it say when you type 'sfdisk -l'?

A network glitch or a firewall that blocks the ssh port, port 4025, or http port could also explain this.

In the next couple of days, I will try backing up and restoring an IPcop VM to see if I can reproduce the problem.

anm said...

Strange - today it worked for linux (IPCop restoration from the same age old image) and NTFS - new backup, restored on new hard disk. Everything worked smoothly. Great work rxhui :)
Though I am not very confident about my earlier problems. I would be working on many different combinations and will keep on filling your comments section.
Thanks!

rxhui said...

I plan on doing a new PBA release 1.1.0 for end of the year, but I won't actually get started on it until mid-November because I have business school applications to complete. i.e. work on MBA before PBA :)

In 1.1.0, this is what I have planned.

- migrate to ubuntu dapper drake
- remove X, don't need it
- multicast
- speed improvements (maybe). I'm looking at speeding up operations that are using ssh.
- options that you can toggle
- misc changes
- improve robustness

I'm open to suggestions.

gmzm said...

I like the sound of this.. I'd love to see the ability to autorun the backup

1. Client PC/Server autoreboots (say 11pm each night)from .iso install on USB flash drive.

2. At completion of backup a flag is set on USB drive to "skip boot" PC/server boots up normally.

3. Upon Reboot the client PC/server deletes the "skip boot" flag from the usb device so that when the automated 11pm autoreboot comes around again another backup is made!

anm said...

Sorry rxhui for making a disappearing act. I have been busy with getting IPCop on the track. I had some success with WinXP and linux backups and restores - seemingly doesn't work all the time. I am sure there is some problem at my end - like a loose IDE cable, or restoring on a computer where hardware is radically different.

Suggestions for the new release:
->Allowing user to select which partitions (or root level directories - or n level) to backup. You may create the same partitions, but do not backup/ restore the not so important partitions. For examples, I may be interested in backing up only the OS + config, but not individual data/ log files.
-> Option to build an ISO with my own prefernces burnt on it.

Good luck for your MBA :)

Breakthrough Tech said...

Backing up SATA drive from pba-client-1.0.1.iso. When I boot from HP 5100 SFF PC with a SATA drive the client fails every time. Is there a place to get a client that works with SATA system?

rxhui said...

There are two clients available for download from sourceforge. The default one is based on Insert Linux, and there's an alternate one based on Slax. You have have better luck with driver support on the Slax CD. Give it a try.

http://sourceforge.net/project/showfiles.php?group_id=17141

SunnyDayz said...

rxhui, i have just stared to use PBA and have just one question. How do i change the default pass. I have changed the user account password but the private shared folder still wants the vmware pass. I have looked around and can not find the awnser to this. Also i have successfully backed up from an IDE drive, then resored to a SATA drive with no problems using PBA. (winxp pro/NTFS)

rxhui said...

To change the system password, use 'passwd', to change the password for the samba shares, use 'smbpasswd'

Anonymous said...

I gotta tell you, I REALLY love PBA! I do occasional tech support on an ad-hock basis, and I love that I can take the PBA VM along on my laptop and crank it up at a support call to image and restore a system! We are looking to use it at work as well (day job) to manage our system images using a VM Server with a big disk array.

You posted on VMTN features that you were thinking of adding. I wanted to get my votes in...

#1 Option to burn system recovery CD or DVD from a backup.
#2 PXE support for booting the client over the network
#3 Server driven backups instead of client driven backups (to use in conjunction with PXE Support)

A system recovery Disk would definitely be #1. Cause I would like to be able to create a custom restore CD for my clients and Work's Laptop users. That way if I am not nearby, or they are out of the office, a restore is as easy as "pop in the CD, select to boot from it, viola!"

From my perspective PXE and Server-side restores is a Very delicious Icing on the cake!

Keep up the good work!

Anonymous said...

I gotta tell you, I REALLY love PBA! I do occasional tech support on an ad-hock basis, and I love that I can take the PBA VM along on my laptop and crank it up at a support call to image and restore a system! We are looking to use it at work as well (day job) to manage our system images using a VM Server with a big disk array.

You posted on VMTN features that you were thinking of adding. I wanted to get my votes in...

#1 Option to burn system recovery CD or DVD from a backup.
#2 PXE support for booting the client over the network
#3 Server driven backups instead of client driven backups (to use in conjunction with PXE Support)

A system recovery Disk would definitely be #1. Cause I would like to be able to create a custom restore CD for my clients and Work's Laptop users. That way if I am not nearby, or they are out of the office, a restore is as easy as "pop in the CD, select to boot from it, viola!"

From my perspective PXE and Server-side restores is a Very delicious Icing on the cake!

Keep up the good work!

Anonymous said...

How do I remove test uploaded images from the vm?

bvwelch said...

This is a cool project. But sadly, the client cd "hangs" during the bootup on my HP Pavillion dv5000 laptop. It says something about "looking for CDROM on /dev/hdc" it hangs there for a long time, then drops me into a minimal shell.

Any suggestions? I'm able to boot a fair number of other Live Linux CDs, maybe I can find one that has all the tools your script needs?

thanks for any suggestions

bvwelch

bvwelch said...

OK, I tried the Slax cd and it works better. Now I need to figure out why pba thinks my C: on my laptop needs chkdsk... Actually I have seen chkdsk problems and that is why I was trying to backup. oh well...
thanks again for a cool set of programs!

Gandalf said...

Is there a guide yet on how to take a backup of a real machine and be able to convert it to a vmware virtual machine?

thanks.
-Bob

rxhui said...

Gandalf,

Sort of. Back in August, I wrote this up.

http://canned-os.blogspot.com/2006/08/p2v-virtualizing-existing-os-install.html

There's a link to it under the Articles section; it's titled "P2V - Virtualization using PBA." I intended it to be a more thorough guide than it is, but it still may be useful. Perhaps, when I have more time, I will update the guide and make it better.

In windows, the big thing to watch out for is the blue screen that occurs because windows isn't happy about not having the vmware scsi drivers present. I cover this in my guide. In linux, the thing to watch out for is changing from IDE to SCSI and vice versa. You might want to also check out VMware's Converter. http://www.vmware.com/products/beta/converter/

igbalonigbanlo said...

Hi,
Great product. However I have one question. I need to virtualize an installation that takes up an entire 250GB of disk space. I intend to use knoppix qparted or some other such utility to partition most of the empty space. I've used the appliance but I don't see an option to either backup select partitions or if the whole drive was backed up an option to restore only select partitions. Can this be done? This way I can shrink this installation into a smaller than 250GB VM.

ciscoblog said...

North American online casino gamblers constitute 48 percent of the overall Online Casinos Extreme gambling business and research indicates that this might shrink to 36 percent by 2010.

ShayDee said...

I noticed that the 2nd VM hdd used to store the backups is only 20GB. How do I increase it?

garibdabal said...

Hello
I tried PBA on desktop machines and it workd. But when I tried on my laptop Compaq 6710b (SATA Drive) Default Client just hangs.
Then I tried Insert Linux and Slax CD, both clients got laptop boot but without network support (ifconfig doesn't show any eth interface) can not ping anything except lo. Any help would be appreciated. Thanks.