Raspberry Pi NAS for Travelers


20140422_183301

Say you have some type of home NAS or drive array which houses valuable information. And let’s say you want to go traveling for a while (as I plan to do), but leave the box in storage. What’s the best way to bring your data with you, or just a bunch of free space for projects?

For most, ditching the drive in your laptop for something which has more capacity is good enough. But what if you wanted to maintain some level of redundancy for your data?

In my case, I was coming from a 3 x 1TB drive configuration in RAID 5 with battery backed write cache (Dell PERC 5/i), running Openfiler with NFS and SMB.

The answer, of course, is Raspberry Pi.

Obviously the trade off is a drastic decrease in speed, but more than worth the effort to put this little setup together. Also, due to the CPU overhead and complexity, I didn’t want to bother with software RAID and decided instead to use lsyncd.

The kit includes:

1 x Raspberry Pi
1 x Adafruit Pi case
2 x Seagate Slim Traveler 2TB USB drive
1 x Pluggable 4 port USB hub
1 x PowerGen 2.4-Amp USB wall charger

Raspberry Pi alone doesn’t supply enough power for both drives, or even one for that matter. The powered USB hub is necessary for reliable operation of both drives.

Before we begin, we’ll need to install a few packages.

root@raspberrypi:/# apt-get install sysstat ntfs-3g lsyncd bc

Then make sure lsyncd starts on boot:

root@raspberrypi:/# update-rc.d lsyncd defaults

When working with a large directory structure as I was, upping the number of inotify watches was necessary to keep the lsyncd service from stopping unexpectedly.

root@raspberrypi:/# echo "fs.inotify.max_user_watches = 65535" >> /etc/sysctl.conf
root@raspberrypi:/# sysctl -p

Now comes the tricky part. When working with USB drives on Raspberry Pi, the first drive you plug in will always become sda, the second sdb, and so forth. We’ll need to mount each drive by ID, as opposed to sdX so that no matter the order they’re plugged in, each drive will always have the same mount point. I created a /media directory with directories usb001 & usb002. These will serve as mount points for each drive. I found the ID by plugging in one drive, then listing out /dev/disk/by-id. (It’s worth noting that I had already connected each drive to a Windows machine and formatted with NTFS.) Once you have the ID, you can assign the block device to a mount point in /etc/fstab. Then do the same for the second drive.

/etc/fstab:

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
/dev/disk/by-id/scsi-SSeagate_Backup+_BK_NA763W70-part1 /media/usb001   ntfs-3g defaults 0       0
/dev/disk/by-id/scsi-SSeagate_Backup+_BK_NA763W8P-part1 /media/usb002   ntfs-3g ro 0    0

In my case, usb002 had all of my data and I didn’t want to risk accidentally syncing it with the empty drive; note the read-only (ro) flag in the options column. Once everything finishes, I’ll remove the flag and specify ‘defaults’ like usb001.

To mount both drives:

root@raspberrypi:/# mount /media/usb001 && mount /media/usb002

/etc/lsyncd.conf:

settings = {
     logfile = "/var/log/lsyncd.log",
     statusfile = "/tmp/lsyncd.status"
}

sync {
     default.rsync,
     delay = 0,
     source = "/media/usb002",
     target = "/media/usb001",
     rsync = {
          archive = true,
          compress = false
     }
}

Start lsyncd:

root@raspberrypi:/# service lsyncd start

Monitor disk utilization:

root@raspberrypi:/# watch -n 1 'iostat -d -x'

Monitor disk space:

root@raspberrypi:/# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.2G  2.5G  4.4G  36% /
/dev/root       7.2G  2.5G  4.4G  36% /
devtmpfs        211M     0  211M   0% /dev
tmpfs            44M  760K   44M   2% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            88M     0   88M   0% /run/shm
/dev/mmcblk0p1   56M   19M   38M  34% /boot
/dev/sda1       1.9T  1.2T  663G  65% /media/usb002
/dev/sdb1       1.9T  168G  1.7T  10% /media/usb001

Here’s a little bash script that spits out a percentage. You may need to modify per your own configuration.

/root/lsyncd-monitor.sh:

#!/bin/bash
sda=`df -h | grep sda | awk '{print $5}' | sed 's/%//g'`
sdb=`df -h | grep sdb | awk '{print $5}' | sed 's/%//g'`
pct=$(echo "scale=1; $sda * 100 / $sdb" | bc)
echo "lsyncd: $pct% complete"

Make it executable and then watch the process:

root@raspberrypi:/# chmod a+x lsyncd-monitor.sh
root@raspberrypi:/# watch -n 1 './lsyncd-monitor.sh'

Once the drives sync up and everything looks good, remove the ‘ro’ attribute in /etc/fstab if you set it previously, and use the source drive as your primary Samba share.

Lsyncd will automatically sync additional files, deletes, etc. with the second drive. If you experience any difficulty with Raspberry Pi, lsyncd, or drive complications, simply disconnect and plug directly into another machine.

 

About Benjamin Perove

Ben has been associated with a broad spectrum of technologies starting from an early age, and he's contributed to the success of many businesses and enterprises since 2001. Most of his time is spent building cool stuff. When he's not working, he enjoys reading, playing acoustic guitar, and being with friends. He currently resides in Chiang Mai, Thailand.

25 Comments so far

  1. Hyshai @ May 14th, 2014

    Is there a reason why you used drives formatted as NTFS, is that a requirement for lsyncd? I’ve had much better performance with ext4 formatted drives, so I’d rather be use NTFS.

  2. Viktor @ May 14th, 2014

    great idea, but i have 2 questions:

    first: why not just use both drives in a software array created at your main Laptop?

    second: what Write or Read speed i have to expect using a Mbit Connection? Did you have measure it?

    greets, Viktor

  3. JB @ May 14th, 2014

    Wish I saw this before I hit the road (7 months now and counting). I had the same situation, leaving a redundant NAS at home. The details of my solution are irrelevant but the one thing I did learn from it is this: HDs will fail.

    I’d love to hear the outcome of using your solution.

  4. […] Raspberry Pi NAS for Travelers | Benjamin Perove […]

  5. Ben @ May 14th, 2014

    Hyshai – NTFS isn’t a requirement of lsyncd. Given that I’m using two large capacity drives (which aren’t exactly known for their performance), ext4 wouldn’t make that much difference. Also, Windows is dysfunctional when it comes to reading anything that isn’t NTFS. If it weren’t for that, I would have formatted them with ext4.

  6. Ben @ May 14th, 2014

    Victor – it would be less of a standalone NAS if I was always having to connect those drives to my laptop. I have two laptops, a phone and a Kindle all connecting to an ASUS RT-AC66U running DD-WRT. Each device except the Kindle mounts a Samba share. My laptop is connected at about 866 megabits; the Pi tops out at 100 megabits. I can get a pretty consistent throughput at 45-50 megabits for both read and write. Suppose I could get a faster WiFi interface for the Pi, but then I’d start to hit the 480 megabit limit of the USB bus.

  7. Ben @ May 14th, 2014

    Hey JB – I’m with you there. I do expect eventual drive failure which is why there are two, but when the time comes, I’ll just buy another and wait for it to sync up again. If I have really bad luck and both go at the same time because of a bad batch, I synced everything with Amazon Glacier before I took off. Worst case scenario – I’d have to restore from there. A lot of my really, really important stuff resides in a TrueCrypt volume on Dropbox. The Pi NAS is a great place for music & videos, screencasting, sound recording, video editing, etc.

  8. why @ May 14th, 2014

    Why aren’t you powering the Pi off the hub?

  9. […] Raspberry Pi NAS for Travelers | Benjamin Perove […]

  10. Brandon @ May 14th, 2014

    Could you have used the powered USB hub as the Pi power supply?

  11. […] Raspberry Pi NAS for Travellers [Benjamin Perove] […]

  12. […] Ben Perove has written a tutorial for setting up a Pi as a NAS (storage) device that is as compact as he can make it. Read more here […]

  13. Ben @ May 15th, 2014

    Brandon – yes that’s what I ended up doing.

    RPi NAS setup

  14. […] in creating your very own Raspberry Pi NAS system? Jump over to the Ben Perove website for full instructions and everything you need to be able to build your very […]

  15. Raymond Day @ May 16th, 2014

    Did you know that USB power supply is 1.2 amp on each of the 2 ports. They say 2.4 amps all the time on things like that to trick you.

    Just plug the Raspberry Pi in the hub and use just one power supply that came with the hub.

  16. AnUser @ May 16th, 2014

    Writing in a ntfs partition on linux requires too many resources. Have you tried an ext4 driver for windows?

    Search for paragon ext4, your security filter thinks that I’m trying to hack you if I post the link.

  17. Jim @ May 26th, 2014

    If your laptop is running Windows 8/8.1 an easier way to do this is to plug your usb drives into a powered hub, plug the hub into your laptop running win8/8.1 and then configure the drives using StorageSpace. You can find storagespace in the control panel and it will even configure your drives as redundant storage.

  18. Sharad @ May 30th, 2014

    Great project Ben. I was thinking of the exact same thing while battling with the Buffalo Linkstation Duo I bought for cheap a few days ago.

    I think the HDD failure is just a thing waiting to happen but then thats what the redundancy is for.

    However, if you setup the fstab with the id of each disk wouldn’t you have to fiddle with it if you swap out with another (new disk)?

    thanks for the intro. I am off to buying a Pi.

  19. Sharad @ May 30th, 2014

    On another thought, I was thinking of using a microATX board with fedora to build this setup. That way I could avoid the external powered USB hub and maybe add a little more oomph with the many WD Blacks I have lying around.

    Thoughts? Comments?

  20. […] Raspberry Pi NAS for travelers. Una Raspberry Pi, un par de discos duros portátiles, algo de electrónica barata… y ya tienes tu NAS! Perfecto para quien viaja mucho y no confía en dejar sus datos en la nube. […]

  21. great post to read @ June 7th, 2014

    great post to read

    benperove.com: Howtos and tutorials for Windows and Linux by Benjamin Perove

  22. Jacksonville Website @ July 3rd, 2014

    Jacksonville Website

    benperove.com: Howtos and tutorials for Windows and Linux by Benjamin Perove

  23. carter baby clothes collection @ August 23rd, 2014

    carter baby clothes collection

    Raspberry Pi NAS for Travelers | benperove.com

  24. Annamarie @ September 28th, 2014

    Annamarie

    Raspberry Pi NAS for Travelers | benperove.com

Leave a reply

Add this site to your Firefox Search Bar

Twitter Activity

Recent Entries

Topics

Archives

Got WordPress Security?

You will definitely wish you had downloaded my top 5 recommendations when you're cleaning out malicious Javascript from deep within you WordPress site. Enter your email and get the PDF right now, before it's too late.