Storage – Netgear ReadyNAS RN 214

I’m not a professional tester of IT equipment. And this blog is primarily about film photography. But I can’t avoid addressing the issue of digital image storage: unless you develop your film in a dark room, use an enlarger and get large prints the good old way, the images on your film will be digitized at some point, will be consumed digitally, and will have to be stored and archived on digital media. Because Adobe Lightroom is more flexible than the proverbial cardboard shoebox.

Over the years, I’ve been using consumer grade storage systems from brands like Buffalo and LaCie, until I settled on a Network Attached Storage system (a NAS) from Netgear. The RN104, that I purchased in 2014, fulfilled his duties honorably until last year, when it started to misbehave: the disks got corrupted  (probably because of an unstable supply of power) and I had to restore the data from a backup on Amazon Glacier. More recently a power spike (probably due to a bad connection between the external power brick and the NAS enclosure itself) fried the motherboard and gave me an opportunity to reconsider my allegiance to Netgear, and to consumer grade NAS in general.

netgear system page
Netgear – the system admin page (overview)
netgear_readynas_214
Netgear ReadyNas 214

What I’m asking is pretty simple: I don’t want to store Terabytes of images on a single laptop equipped with a single drive – I want to store my pictures (RAW and jPEG) on a device accessible by the computers (PC, Mac, iPad) connected to my wireless LAN. That device has to be local – I’m using Lightroom to catalog, upload, edit and print my images, and a broadband connection would be far too slow if I used some form of cloud storage as the primary location of my pictures. That perfect device should also act as a Time Machine target for the backup of my Macs. And of course, because hard drives are inherently fragile, I want the device to offer some form of disk redundancy – ideally, it should be also be able to backup its data on a low cost, on-line archival service.

Most of the recent NAS devices (from Netgear and from competitors) meet those basic requirements. They can also stream video (they take care of decoding) and, because their OS is generally based on some Linux distribution, they can be used as multi-purpose servers (not only as file servers, but also as application servers to run Drupal, Joomla, php, or Python applications, for instance). I have no use for those features and they’ve not been part of my evaluation criteria.

netgear_apps
Netgear – some of the applications that can run on the system
  • So, what I’m looking for?
    • a NAS,
    • solidly built (case, power supply, connector)
    • with removable drives – that can be moved to a SAN enclosure of the same family, without losing configuration or data (in case the original enclosure dies, or a capacity upgrade is necessary)
    • with good data protection (RAID 5 or better)

The Netgear RN104 met the requirements for the most part:

  • most of the issues I have encountered with my old RN104 have been power supply and power supply connector related – with dreadful consequences for the data on the disks and ultimately for the chassis itself.
  • until the big crash last year, the NAS was configured with RAID-X, the proprietary implementation of RAID in Netgear’s devices. With RAID-X one disk is reserved for parity, the other disks store data (it’s more or less equivalent to RAID 4). With Raid-X, volumes are easy to expand, but if you lose more than one disk, you’re dead in the water.
  • the Western Digital RED 1TB disks  that I bought separately for it (the Netgear chassis can be purchased diskless) proved flawless

When the RN104 chassis finally died, I considered buying a Sinology or Qnap enclosure, but I would have had to reformat the drives and restore everything from the Amazon Glacier backup, again. Sinology and Qnap are well considered on the marketplace, but seem to use the same type of external power supply brick as the Netgear, and maybe even the same dreaded power connectorUnfortunately, chassis with a built in power supply are much more expensive. 

netgear volumes page
Netgear – all 4 disks healthy – it’s configured with Raid 6 (too many drawbacks with X-RAID in comparison)

Ultimately, buying a new Netgear NAS device appeared to be the lesser evil. The RN214 unit I bought accepted my old Western Digital drives and recovered its configuration automatically from them. It was on line in less than 15 min after I had received it. Performance seems to have massively improved during the last 5 years. The new units  have a quad core ARM processor at 1.4 GHZ and 2GB  RAM, as opposed to a single core processor at 1.2 GHz and 512 MB RAM for old  model. The power brick and the connector are the same, but being new, everything clicks reassuringly and I hope they will age better than their predecessor.

netgear services
Netgear – services enabled on my system

The operating system is the same  as before (Netgear 6.10), the unit accepts the same additional applications (plus a video streaming app that could not have worked on the old unit). As before, the unit can connect to a few external cloud storage services to backup its data (but not to Amazon Glacier, unfortunately) and the Web user interface is reasonably pleasant to use. I did not have to configure this unit (the config information is stored on the disks and moves to a new enclosure when you swap the drives) but my recommendation would be to read the manual carefully if you want to configure a unit from scratch (the factory defaults are not always the best, in my opinion).

I paid $250.00 for the diskless unit (it’s discounted at the moment). Netgear also offers models pre-populated with disks. There is a good warranty on the hardware, but tech support is only available as an extra-cost subscription (storage issues can be vexing,  hard to diagnose and time consuming to fix, and I understand Netgear can’t offer free support on a device  sold for a few hundred dollars). But Tech Support won’t get your data back if your disks are too badly corrupted, so a good backup is your best friend.

Back to photography, now…


PA14161144816
One of the oldest pictures on the Netgear – Shot in 2002 – scanned and copied from system to system ever since – Pornic – France – Minolta Vectis S1

 

 

 

Amazon Glacier – an archival solution for your digital memories

One of the biggest challenges of digital photography is the long term archival of the images. And because slides and negatives are generally scanned, and end up in the same post-processing chain as “native” digital images, they’re subject to more or less the same issues (I guess that you could still go back to the original negative or the slide and re-scan it, but you would have to locate it first).

lightroom
This early digital picture was taken in 2002 with a Samsung digicam and stored in iPhoto. But it was imported in Lightroom at a later date and is still accessible.

There are three big obstacles to the long term preservation of pictures in a digital world:

  • the long term availability of the digital asset management software,
  • the evolving file format standards
  • the inherent fragility of the medium used for storage

Users of the original version of Apple iPhoto, of Apple Aperture, of Microsoft Expression Media and of plenty of other discontinued products have not lost the original images stored in their photo libraries, but they have lost an easy way to access them – and in some case, of all the changes and adjustments (crops, exposure, contrast and curves) they had performed. Of course, it’s always possible to port the images to … the standard of the moment: Adobe Lightroom, but it may require a serious effort.

Adobe Lightroom is not about to disappear (on the contrary, it has become a de facto monopoly), but Adobe may progressively price it out of the reach of amateurs: they have already transitioned to a subscription-only licensing model, which may make sense for professionals, but is costly for amateurs who used to perform an upgrade every 5 years or so…

Surprisingly, evolving standards have not been too much of an issue so far – after early challenges by patent trolls were defeated, JPEG has led a quiet life. Evolutions of JPEG are being discussed in the international standardization bodies, but they promise to maintain backwards compatibility. At this stage, jpeg is still jpeg, tiff is still tiff, and we can still read files saved 15 years ago.

The proliferation of RAW file formats (how many for Nikon or Canon already? ) is also a potential issue, but computer Operating Systems and RAW converters still keep up – and support most of the old RAW formats, even though it’s probably wise to keep a JPEG or a DNG version of your images, just in case.

readynas_working
A NAS in working order (here, a Netgear Readynas with 4 1TB drives, all up) and Raid 6 configured.

Which brings us to the worst issue by far – the medium (tape, CD, DVD, hard drive, cloud blob) used for storage.

  • the storage needs have exploded (24 Mpixel is the new normal, and I know amateurs who refuse to shoot with anything less than a 40 Mpixel camera, like the Pros) – shooting 10 Gigabytes worth of images per day has nothing exceptional anymore,
  • At the same time, the capacity of WORM devices (CD, DVD, …) has stagnated,
  • solid state media is still expensive,
  • spinning hard drives have capacity but are fragile,
  • in spite of all the promises, consumer grade Network Attached Storage (NAS) is far from 100% reliable,
  • on line backup/archival services and cloud hosting services come and go (many vendors have decided to leave the consumer market, while some services are tied to a specific brand of computer or smartphone hardware), and some free photo sharing services may sell your secrets to advertisers (“if you’re not paying for the product, you’re the product”).
corruped_pict
Images can also get corrupted – without a good backup, the image would be lost forever (Lightroom does not store the images in its catalog, just the metadata and the “development” instructions, the issue is with the NAS or with the file sharing protocol).

For  long term storage at home, hard drives are currently the best option, but at least in my case,  they’ve been quite unreliable: over the last 10 years,

  • I lost two hard drives on my personal laptop, before I upgraded to a SSD – which has less capacity but seems to fare much better when it comes to reliability,
  • I lost a hard drive on the Apple Time Capsule I was using for backups (Green Seagate Barracuda)
  • I lost a LaCie network attached hard drive (a Barracuda also, I’m afraid)
  • Files got corrupted (see above),
  • the Netgear ReadyNAS RN104 (with four 1 TB drives arranged in a so-called X-RAID) lost its file allocation tables (even if the Western Digital Red disks were still OK) and had to be reinstalled from scratch – without using X-RAID this time, but under a proper RAID 6 scheme instead.
Netgear issue
The dreaded Netgear error message – search “ReadyNAS – Remove inactive volumes to use the disk. Disk #1,2” on Google to see other examples (source: Netgear Communities Forum)

Fortunately, I’ve always had relatively good backups (not 100% success at recovery – there’s always something that falls thru the cracks, but close enough)

Here is how my pictures are processed and protected, currently:

  • if I’m using a modern digital camera:
    • while traveling – I upload the files to my iPhone over Wi-fi at least once a day – then Apple syncs it to my Photo library in iCloud. It’s not a full backup – my Fujifilm XT-1 camera only uploads JPEG files via Wi-Fi, it does not upload the RAW files, and with a resolution limited to 1776×1184 (a bit above 2 Mpixels)  – but it’s convenient, good enough for social network updates, and better than nothing if the SD card fails or the camera is stolen,
    • the “exposed” SD or CF card are copied as soon as possible to the SSD of a  laptop;
    • and I store the SDs for up to 6 month before reformating and reusing them.
  • if I’m shooting with film:
    • I don’t have any form of backup until the film has been sent to the lab, processed and scanned (it’s the rule of the game with film – but it always makes me uneasy when a drop an envelope with a few irreplaceable rolls of film in a USPS mailbox, even if they have a 100% reliability record with me so far).
    • when the scans are available, I download them to the SSD  of the laptop, and  when I receive the negatives from the lab, I keep them in the proverbial shoebox.
  • once the JPEGs, the RAWs and the scans are on the laptop,
    • there is an automatic backup process to an external HDD drive (using Apple TimeMachine), to the NAS (TimeMachine again), and to Amazon Glacier (using the ARQ backup application)
    • I upload the pictures to the Netgear NAS for Lightroom processing and archival,
    • and the Netgear NAS is backed up to Amazon Glacier using the ARQ client of the Mac.
arq restore
ARQ backup – the restore request is being processed by Glacier. In the background, Lightroom with the folders already restored on the ReadyNAS.

Amazon Glacier

  • Amazon Glacier is the long term archival service of AWS (the Amazon Cloud). Storage is extremely cheap ($0.004 per GB per month) and Amazon keeps multiple encrypted copies of the data in multiple AWS data centers.
  • There are all sorts of interesting features for Enterprise clients. But it’s not the exclusive domain of IT departments and the man in the street can also store files on Amazon Glacier.
  • Now there’s a catch: data retrieval is not instantaneous (Amazon needs 3 to 5 hours to start processing the request in the standard retrieval mode) and it’s not free either ($0.01 per Gbyte in the standard mode) – which is perfectly fine if you remember that Glacier is about long term storage. Consider the typical use cases for an amateur photographer:
    • you lost the pictures of that fantastic trip you made 10 years ago – it’s not going to be an issue for you if Glacier starts retrieving the pictures 5 hours from now,
    • you lost a hard drive and its local backup with 1 TB of pictures (to a flood, a fire, a burglary, a massive power surge) – again, you’re not going to complain if the data retrieval actually starts a few hours after you requested it: you’ll be happy to retrieve  your files, even if it takes time (assuming 1TB, that would be 44 hours on a 50 Mbits broadband circuit continuously operating at that speed, which means much more time in reality) and you will have to pay a few dozens of dollars for the service.

Arq

  • Arq is a backup solution for Mac OS and for Windows, leveraging the storage and archival services provided by a large selection of public cloud services. I’ve been using it in conjunction with Glacier for a few years, and it’s proved its worth a few times already.

It may seem like overkill – but massive hardware failures, catastrophic events and user errors happen, sooner or later. If you don’t want to lose your pictures eventually, do something, now.


Definitions, Buzzwords and Acronyms:

Archive: collection of records kept for long term retention. Typically, archives are not actively used.

Backup: “process of making extra copies of data, that will be used to restore the original in case it is lost or corrupted”

AWS: Amazon Web Services – the on-demand cloud computing platform of Amazon.com

Cloud (cloud computing): Cloud computing is shared pools of configurable computer system resources and higher-level services that can be rapidly provisioned with minimal management effort, often over the Internet. Cloud computing relies on sharing of resources to achieve coherence and economies of scale, similar to a public utility. (Wikipedia)

HDD: hard disk drive – they’re called hard disk drives because there are made of a few hard, metallic disks spinning at high speed, with tiny mechanical arms moving a magnetic head a few microns above the disks. The technology has been here forever, hard drives are cheap, offer a large capacity, but are somehow unreliable over the long run. (see Backup, above)

NAS (NAS Drive): Network Attached Storage – appliance containing one or more hard drives, connected to a LAN, that provides file level data storage to PC or Mac clients. Practically, a NAS is a small file server, generally running a version of Linux, with an easy to use Web based configuration interface. For the user of a PC or a Mac, the NAS just presents itself as another storage volume in Windows Explorer or in the finder. Models supporting two or more disk drives generally offer redundancy mechanisms (mirroring, RAID) to minimize the consequences of a hard drive failure.

RAID: (Redundant Array of Independent Disks): a technology that provides data redundancy and performance improvements in storage systems using multiple physical disk drives. Having a NAS configured with RAID is not the panacea and does not dispense from running regular backups: RAID usually protects the data if one disk fails, but it does not protect against a massive failure (two or more disks fail, a disk controller corrupts the data) or against human error (files erased by mistake).

SSD: solid state drives. With a SSD, information is stored on microchips. There is no moving part. SSDs are both faster and more expensive than Hard Drives, that’s why they are used in laptops, but not in long term storage systems.


69320016
An image restored from a backup – Atlanta – Nikon FM – Nikon 24mm AF