Storage Considerations for Photographers

Many photographers and videographers often rely on using local storage within their computers to store media files. Once their computer drives fill up, they often end up purchasing external hard drives to offload some or all of the data, segregating the data and making it difficult for quick access and backup. Others choose dedicated storage arrays that handle all the data in a single location, albeit at a higher cost and sometimes questionable results. For a person who is not technology-savvy, choosing a proper storage solution can be a difficult task. With so many different DAS (direct-attached storage) and NAS (network-attached storage) solutions on the market, one can get quickly confused, ending up with many storage issues. Since mismanaging storage can potentially lead to data loss and other problems, it is always a good idea to revisit your storage needs and choose the best solution that will not only address the storage requirements you have today, but also help with growing your data in the future. In this article, we will go over different storage solutions in detail, identify pros and cons of each and hopefully help you in selecting the best storage solution for your needs.

1) Handling Data Growth

Year after year, the resolution of both recording and output devices is increasing at a significant rate. While a few years back we used to deal with cameras that had at most 8-12 megapixels (MP) of resolution and typical monitors / TVs did not exceed 1080p, things have certainly changed since then. Common camera resolution has been bumped up to 20-24 MP now (with high resolution full-frame cameras pushing 50 MP), whereas most electronics stores don’t even stock 1080p screens anymore, with both TVs and monitors getting pushed to 4K and higher resolution. As a result of these changes, media size has also seen a significant increase. In just a matter of a couple of years RAW files quadrupled in size from 8-12 megabytes (MB) to 40-50 MB on average when shooting with 36 MP+ digital cameras, and 4K video presents a new level of storage challenges, not just in terms of storage space, but also in terms of storage speed to keep up with the large amount of flowing data. While you can do your best to reduce your backup needs, the larger files and the additional data you have to deal with on a daily basis do not make it easy to keep your storage needs minimal.

All this data growth makes it difficult to keep media stored on a single laptop or a computer, so most photographers and videographers end up purchasing large capacity external drives where they offload data. Once those drives fill up or a new year rolls in, they typically buy more drives to accommodate their new data, ending up with a collection of external drives that become a nightmare to manage. Since the data is segregated across many different drives, accessing older data becomes a hassle, performing routine backups gets difficult and the sheer number of different devices with different capacities and connectivity options make it difficult to manage data as a whole. And let’s not forget about the typically miserable speed of accessing data from external devices. Sounds familiar? It sure does to me, as I used to do it myself in the past. And I know a number of professional photographers who still rely on such workflow practices.

What if there was a way to streamline the process by putting all the data into a single location? Wouldn’t it make it easier to quickly access and manage the data? That’s where storage arrays come into play.

2) What is a Storage Array?

In simple terms, a storage array (also known as a disk array) is a dedicated box that can accommodate multiple storage drives (typically hard drives or SSD drives) to store and access data. The simplest and the least expensive storage arrays typically accommodate two to four drives, whereas higher-end storage arrays can accommodate five or more drives and offer fast speeds, more connectivity options and might even run their own operating system to handle different native and third party applications. Unlike typical local and external storage, higher-end storage arrays often have built-in error-detection and redundancy features to reduce the potential for data loss, making them more reliable and scalable in the long run.

Below are the major storage architectures you might come across today:

• Direct-Attached Storage (DAS): any storage that requires a direct connection to a computer can be considered a DAS. For example, your hard drive or SSD drive on your computer is a DAS. An external drive or storage array connected with a Thunderbolt or a USB cable to a computer is also a DAS. DAS is typically only beneficial for a single computer setup and it is usually not designed to run autonomously. DAS provides “RAW” or block-level storage to a computer, so its storage volumes are typically managed through the computer it is connected to.

  

• Network-Attached Storage (NAS): as the name implies, network-attached storage connects to one or more computers via wired or wireless network (although wireless network connectivity options are typically not available due to wireless reliability issues). Since NAS is not directly connected to a computer, it requires its own CPU and memory in order to manage its storage and hence it is designed to run autonomously. NAS is ideal for a multi-user setup and as the next section reveals, has many advantages over DAS, especially when it comes to data access. Unlike DAS, NAS provides file-level storage that can be accessed from multiple computers at the same time.

  

• Storage Area Networks (SAN): storage area networks are more complex than DAS and NAS, and they are typically utilized in larger environments that need high-level performance, reliability, scalability and redundancy in datacenter environments, where a single large storage array can be used to provide block-level storage to multiple servers. Since SAN is overkill for most photographers, we will keep it out of scope of this article.

Depending on the architecture, storage arrays can be simple and can also be very complex. Simple, user-friendly storage arrays are typically going to be the most affordable, giving very few configuration options to quickly get started, whereas storage arrays designed for small to medium-size businesses can have many configuration and connectivity options, making them more difficult to implement and manage.

3) DAS vs NAS

Let’s take a look at the advantages and disadvantages of both DAS and NAS:

DAS Advantages:

• Simple to Set Up: DAS is simple and everyone gets it. It acts as a drive on your computer. You can easily format it, resize it and partition it with standard operating system tools. It does not require a network or any other extra configuration.
• Excellent Speed: When connected via Thunderbolt or USB 3.1, DAS can provide incredible speeds, taking advantage of everything the storage has to offer.
• Better Latency: A direct connection is always going to have less latency than a network connection.
• Low Cost: While there are some expensive DAS storage arrays out there, most DAS solutions start out really cheap. A simple hard drive in an enclosure is a DAS and you can even get a cheap DAS setup with two drives that can provide redundancy in RAID 1 (mirror) configuration.
• Ideal for Travel: For those who often travel, a simple DAS enclosure with one or two drives is ideal, since it takes up very little space.

DAS Disadvantages:

• Single Computer Only: DAS must be connected to one computer and its storage cannot be shared with multiple computers with multiple cables. Once connected, the storage can be shared with multiple computers over network through the computer’s operating system, but it requires the computer DAS is connected to be always on, which increases power usage.
• Limited Options: Due to DAS always requiring a computer connection, it can be rather limited when it comes to things like external access. Instead of providing access to the storage array exclusively, one has to provide access to the computer DAS is connected to and set up a web or file server, which is a rather technical task. The DAS cannot be managed over the network, so if anything goes wrong, one must be able to access the computer to manage the DAS.
• Potential Connectivity Issues: In order to be able to get the best performance, a DAS must have a fast controller + connection port on its side and the controller and port speed must match it on the computer, or the speed will be limited to the slowest connection. For example, if a USB 3.1 DAS is connected to a USB 3.0 port, it will slow down to half its speed. And if it is connected to the old USB 2.0 port, it will slow down to a crawl, leading to many frustrations. In addition, due to the frequent use of DAS devices, the process of connecting and disconnecting interface cables can lead to damage of cables and interface ports.
• Risk of Data Loss on Simple DAS: When using a simple DAS with a single drive (one local drive or one external drive), there is a high chance of data loss in case the drive fails. Ideally, if one uses a DAS, it should be set up with at least two drives in RAID 1 / mirror configuration.

NAS Advantages:

• Ideal for Multi-User Environment: One of the biggest benefits of network-attached storage, is that you can set it up for a multi-user setup. So if you have multiple computers on your network and you want to be able to reliably access the same storage from them without requiring one computer to always share a connection, you need a NAS.
• Power Efficient: Since a dedicated NAS device has its own CPU and memory, manufacturers often use very power efficient processors and memory in order to reduce energy usage of the NAS device. While with a DAS you will be burning through a lot of power (computer power + DAS power), you will be saving quite a bit of money by going with a power-efficient NAS.
• Better Redundancy: Most NAS arrays have built-in RAID controllers that provide excellent error-checking and redundancy options.
• Dedicated for Storage: A NAS is a dedicated storage device that does nothing except providing storage for computers and other devices on the network. Because of this, it can be optimized to provide fast, reliable and scalable solution that can meet many storage demands.
• Isolated Users and Permissions: Since NAS works autonomously, it can be set up with a different user and permission structure than your computer(s), keeping separate security and making it easy to grant access to particular folders.
• Built-in Apps: Another great benefit of a NAS is that it comes with a set of native built-in apps that extends the functionality of the device and depending on its popularity, there might even be third party apps available for you to deploy. For example, you could easily turn a NAS into an entertainment device that streams audio and video content to TVs and other devices on your network. You could set up a NAS to automatically perform backups on your computers. You could set up a file server, a web server or even a telephone system.

NAS Disadvantages:

• More Complex to Set Up: Compared to DAS, NAS is more complex to set up and configure. One needs to have a properly set up network, understand networking basics (such as assigning an IP address to a device) and have matching interface speeds on all devices and cables. For example, if a NAS device has 1 Gbit network connection, all cables, network cards and switches on the network that require 1 Gbit access must match that speed.
• Network Throughput / Speed Issues: While modern DAS interfaces are optimized for speed to deliver 10 Gbit+ throughputs, modern networks have not been keeping up with the same pace. Even the latest generation laptops and computers are shipped with Ethernet ports limited to 1 Gbit throughput and most consumer-grade NAS devices are also limited to 1 Gbit ports. While manufacturers like QNAP and Synology ship their NAS devices with at least two ports to be able to allow for more network throughput via link-aggregation protocols, such setups require specific network switches and can be difficult to configure.
• Slower Latency: Since all the data flows over the network, there might be issues with latency, especially if the network connection is already in heavy use.

As you can see, one is not necessarily better than the other – both DAS and NAS devices and storage arrays have their uses and their pros and cons. So before you make your purchase decision, you should understand them fully and be ready to deal with the potential issues you might encounter.

4) DAS and NAS Speed Considerations

While direct-attached storage arrays are capable of delivering exceptional speeds (provided that one uses a fast interface such as USB 3.1 or Thunderbolt, as stated above), the biggest speed bottleneck of network-attached storage is the network. With most homes and offices wired with older CAT-5e cabling that limits network throughput to 1 Gbit, running a significantly faster network not only requires expensive switches and network cards, but also potentially new cabling, which can be very cost-prohibitive to replace (please note that CAT-5e and CAT-6 cables can potentially support 10 GbE at shorter distances).

One could argue that a DAS array would be a better solution, since it can handle a lot more throughput. However, NAS arrays have a lot of advantages over DAS arrays, especially in a multi-user environment. With a NAS device, your media can be accessed from any computer on your local area network (LAN) and if you need access to your files when traveling, you can download your data over the Internet as well, without having to keep one computer running all the time. NAS consumes a lot less power than a DAS connected to a computer and one can use all kinds of native and third party apps to get more functionality from a NAS, such as automated computer backups, automatic replication to other NAS devices and ability to stream media to home entertainment devices. Lastly, most NAS arrays are built to work 24/7 and are self-managed, whereas consumer-grade DAS arrays are usually not.

For these reasons, I highly prefer NAS to DAS as my storage solution to store photographs and video. While I have been quite happy with my Synology DS1815+ for the past few years, unfortunately, I cannot copy media faster than 100-120 MB/sec from a single computer due to the 1 GbE network interface limitation, even with link-aggregation enabled (link-aggregation can only handle more data when data is transferred over multiple computers). And since I often work with large RAW / TIFF files and video, plus my storage needs are only increasing overtime, the network bottleneck has definitely been a limiting factor. For this reason, despite the high cost of implementation, I have already moved up my network to 10 Gbit Ethernet and started migrating my data to the latest NAS models from QNAP and Synology with 10 GbE network interfaces.

So if you have been trying to decide on a NAS solution, do keep the above-mentioned network bottleneck issues in mind and make sure to plan for the future. If you work with 4K+ video content or have specific speed requirements and you do not need more than one machine accessing your storage, a high-performance DAS might be a better solution. And if you have any interest in pursuing a 10 Gbit Ethernet NAS setup, I will be working on a separate article with all the details next.

5) The Need for Larger and Faster Storage

Thankfully, with the growth of photography and videography material, storage manufacturers have also been working on increasing storage capacity. If a few years ago we were limited to 4 TB drives, there are now 10 TB consumer-grade hard drives available. Although the cost per GB still remains high, M.2 NVMe and SSD drives are also seeing increases in capacity, with 4 TB+ options already available on the market. As a result, we now have storage arrays that can deliver extremely large capacity volumes capable of handling a lot of data. And with the added benefit of creating large caches based on M.2 and SSD drives, one can push incredible speeds on such arrays as well. Lastly, thanks to the mix of different form factors and types of storage devices, we are now beginning to see tiered storage coming to consumer market, which is very exciting!

Let’s now take a look at some of the best storage solutions available on the market that we recommend to our readers.

6) DAS Recommendations

Let’s get started with the DAS storage arrays that we recommend to our readers. If you are just starting out and you do not want to spend a lot of money on an external drive, I would recommend to get a two-bay DAS / external drive. You can buy those pre-configured with capacities up to 20 TB (in RAID 0 / stripe mode). Personally, unless you know what you are doing and you have your backup fully figured out and in place, I would not recommend to run any storage in RAID 0 configuration. So as soon as you receive yours, I would change it to RAID 1 (mirror) mode, so that if one of the drives were to fail, you would still not lose the data. However, it is important to point out that RAID can never be considered to be a backup, so don’t just assume that your data is safe if you run a simple two disk array in mirror mode. Always make sure that you have a solid photography backup workflow in place. You could still lose all of your data if two disks were to fail at the same time, if the DAS was stolen or some other disaster took place.

So which specific DAS units would we recommend? Here is our list of favorites:

• G-Technology G-RAID 2-Bay DAS Array – Wonderful DAS arrays for those who are starting out. These units have 2x Thunderbolt 2 and 1x USB 3.0 Ports and are both PC and Mac Compatible. They come with a hardware RAID controller and they also look great! Another option is LaCie’s 2big 2-Bay DAS Array, which provides very similar features at competitive prices.
• OWC ThunderBay 4-Bay DAS Array – If you are a Mac user or you have a Thunderbolt adapter on your PC, the 4-bay storage array from OWC delivers excellent performance.
• LaCie 6big 6-Bay DAS Array – if you want to step up big time in performance and total storage space, the LaCie 6big DAS is hard to beat.

7) NAS Recommendations

For the past few years, I have been relying heavily on Synology NAS arrays, which deliver superb performance, sleek and easy to use graphical user interface and excellent app ecosystem. QNAP storage arrays are new to my home as of this year and they also seem to be very solid, reliable and in some cases even offer more features than Synology. Although I have used a number of different NAS solutions in the past from a number of different manufacturers such as Drobo, Netgear and Promise Technology, I have had either bad or very mixed experience with them, bringing Synology and QNAP to the top of my list of favorites:

• ioSafe 214 2-bay NAS – Based on Synology DSM, this 2-bay enclosure is what I have in my basement as my secondary backup. This little guy is fireproof and waterproof, so if you put two large capacity drives in it (I have 2x 10 TB drives in RAID 0 configuration), it can keep a backup of your data. I have it configured so that it automatically replicates changes from my Synology drive, which works great.
• Synology DS1517+ 5-Bay NAS – This 5-Bay NAS unit is ideal for most photographers out there. It can accommodate up to 5x 10 TB drives. If you were to run it in RAID 5 configuration, you would end up with a total of 40 TB of available space, whereas RAID 6 would result in a 30 TB volume.
• Synology DS1817+ 8-Bay NAS – If you have a lot of data, the Synology DS1817+ can accommodate up to 8x 10 TB drives, giving you 60 TB of total storage in RAID 6 configuration. This is what I currently have with a 2x 10 Gbit network card to take a full advantage of its insane speed. See our Synology DS1817+ announcement article for more details.
• QNAP NAS – Similar to Synology, QNAP also offers a number of different products to suit every need. I have to say though, that their QNAP TVS-882T is an absolute monster. I have it running in a production environment with 5x 8 TB 7200 RPM hard drives, 2x 2 TB SSD drives in a tiered configuration and it screams on my 10 GbE network. It is not as energy efficient as the Synology DS1817+, but if you need the best performance with amazing flexibility, the QNAP TVS-882T is currently my favorite of the bunch. Lastly, unlike Synology storage arrays, QNAP can serve as DAS, NAS and SAN at the same time, on top of being able to output content to your monitor or TV.

I will be publishing the review of the QNAP TVS-882T later this week, with the Synology DS1817+ to follow.

8) Hard Drive Recommendations

When it comes to selecting hard drives, there are a number of great options out there. Always make sure to select drives that specifically say “NAS” on them, since they are built for 24/7 operation. Regular drives will work, but their failure rates might be much higher and that’s not what you want when continuously running drives. If you can get a hold of enterprise-grade NAS drives from any manufacturer, those would be the most reliable. However, they are also quite pricey compared to regular NAS drives.

Personally, I have been a huge fan of HGST NAS drives. I have been using them for 5-7 years now and I have never seen one fail. My personal pick would be the HGST 8 TB 7200 RPM NAS drive, which is what I use on my primary storage. They are very fast drives and they are my top choice for NAS storage. While my luck with Seagate has not been that great, the new generation Seagate IronWolf drives look great. The highest capacity drive at the moment is the Seagate 10 TB IronWolf 7200 RPM NAS drive, which is what I would pick if you want to maximize total storage space.

Western Digital also has some great options. The WD 10 TB NAS drive is limited to 5400 RPM, so it should be quieter and not as hot as 7200 RPM drives. It promises the same 210 MB/sec sustained data transfer rate as the IronWolf drives, although I am sure there will be differences when it comes to random IO, with 7200 RPM drives winning over. If you want to step up to 7200 RPM, the only option from WD is enterprise-grade WD 10 TB Gold 7200 RPM NAS drive, which is going to be more reliable in the long term.

Hope you found this article useful. If you have any questions regarding storage, please let me know in the comments section below!

The post Storage Considerations for Photographers appeared first on Photography Life.

Photography Life