This is my rig: Bill Howard’s million-photo $10,000 digital photography workstation
The quest for my new rig started with the need for more speed and fewer crashes. No surprise there. I wanted to edit photos without the lag as Photoshop ambled to life and pulled a big photo off a slow hard drive, or the 20-second wait for Photoshop to blur the background of a photo. I’d do almost anything — buy almost any CPU or SSD — to avoid the annoying pauses that happen a hundreds of times a day.
I wound up building a rig with a six-core CPU on a workstation-class motherboard, dual SSDs and a four-drive RAID array, a single graphics card, and dual monitors. It has a keyboard so powerful it needs two USB cables, and then three scanners and an eight-bay backup server. The rig stores a million photos inside the system unit, twice that on a NAS.
Some of this may sound like overkill. It is and it isn’t. Here’s why.
Wanted: A PC that executes in the blink of an eye
Previously I was working with a four-year-old PC — upgraded but still slow — and a three-year-old iMac. A refresh was overdue. The main PC was painfully slow handling photos, worse as I’ve worked my way into video editing. GoPro video cameras and DSLRs may be small but they churn out a prodigious amount of data.
My goal was blazing fast speed for photos and secondarily for video. How fast? Intel founder Andy Grove said of benchmarks, “Fast is when something happens in the blink of an eye.” In other words, like obscenity, you’ll know fast when you see it. I wanted to be able to instantly flip through 100 full-screen images with no hesitation. I wanted to render a blurred background in five seconds not 15 or 20. I didn’t want to wait 45 minutes for Lightroom to import 1000 photos and create full-size previews.
A million photos takes up 6TB-8TB, so I wanted 8TB, minimum, of internal storage for direct access to all of them. I also wanted data redundancy onboard, meaning some for of RAID. I wanted twice as much backup storage on a server. I wanted a system that stayed cool, a comfortable keyboard and mouse since I’m also a writer, and videoconferencing for talking to clients, friends, and organizations that have discovered Skype and Google Hangouts.
My research included checking with tech-savvy friends in the industry and at ExtremeTech. Here are recommendations I heard:
- SSD is mandatory. Get a second SSD as a hard disk cache or data drive, but you need an SSD for booting up.
- Load up on RAM. With 16GB-32GB of RAM, no Photoshop project will spill out of RAM to a scratch disk and a cause of lag on hard drive systems.
- For Photoshop, the number of processor cores matters more than CPU speed. Performance improves noticeably with 4-6 cores. Beyond six, price goes up faster than performance. Also, in addition to Intel’s Core i7, consider Xeon.
- For Adobe software, a midrange graphics card works well. Just make sure it supports OpenGL.
- Don’t skimp on backup. You need a couple external drives or a NAS with room to store at least twice the data you have today.
With that in mind, I went to work. I actually started building two rigs since I needed to update two systems. I built one near-ultimate system and one bang-for-the-buck system. (More on the second system at the end.)
CPU: Intel Core i7-3930K CPU
My main apps, Photoshop and Lightroom, are heavily multithreaded, meaning the CPU can execute multiple tasks simultaneously within the same program. That means they benefit from lots of physical CPU cores. The newer Ivy Bridge and Haswell architectures focus on dual- and quad-core CPUs. So I went with a more established (read: older) CPU technology with six physical cores and 12 threads: the Intel Core i7-3930K Sandy Bridge-E using the LGA 2011 socket ($550). The K suffix means it’s unlocked and can be overclocked to 3.8GHz.
There was little need for the sibling Core i7-4960 for gamers, almost twice the price to step up from 3.2 to 3.6 GHz base clock speed. PCMag.com said in its Core i7-3930K review, “For all but the most rabid enthusiasts, Intel’s Core i7-3930K processor represents an unparalleled combination of price and performance for use on the X79 Express platform.” This fit with my belief that you often want to shop one model down from the top.
Atop the CPU, I mounted a Corsair H100i liquid CPU cooler ($110). Its dual-fan radiator replaces one of the case fans. The two hoses obstruct motherboard access less than a cheaper CPU-mounted fan but it’s tougher to mount. It uses an internal USB connector for power.
Next page: Motherboard, RAM, and graphics
Motherboard and RAM: Asus P9X79-E WS with 32GB RAM
The Core-i7 CPU was mounted in to an Asus P9X79-E WS motherboard ($460) built around the Intel X79 Express chipset and designed to be the pinnacle of single-socket PC performance. This one has seven full-length PCI Express slots. There are 10 SATA ports with software RAID capabilities but there are limitations if you use SSD acceleration. A heat sink on the board is the size of a helipad.
The graphical UEFI BIOS display is useful and close to idiot-proof. BIOS flashback lets you upgrade the BIOS even if the PC won’t boot: Download the new BIOS from Asus to a USB key, plug that into the white USB jack, press the BIOS button on back, and you’ve got the most current BIOS loaded.
The back panel has a dual gigabit LAN connectors, E-SATA, USB 3.0, USB 2.0. 1394 (Firewire) is on the motherboard too. Audio jacks are based on the Realtek ALC 1150 chipset, more music quality than you’d think techies deserve, unless you’re the user dubbing music onto a video. The one thing missing is Thunderbolt. I fretted over Thunderbolt before getting the board, but not once since.
The P9X79-E WS is an E-ATX (extended depth) motherboard, also called SSI CEB. The front-to-back dimension is an inch deeper than the ATX-standard of 12×9.6 inches. Asus applies the “WS” workstation suffix to motherboards meant to be rock solid, high-performance and things of beauty, that also means they will cost you a fair bit more than the $99 mobo special.
If you’re looking to do a build with a different CPU, you’ll get many of the same features with other Asus MBs ending in WS. The newer Asus Z87-WS, for example, supports the Intel Core i7-4770K, 4771, and 4790 (Haswell) CPU with the LGA 1150 socket and Intel Z87 chipset. The P9D WS is for Intel E3-1200 (Xeon) and Core 3 CPUs, also LGA 1150, using the Intel C226 chipset.
The P9X79-E WS uses a quad-channel memory architecture, so it’s best to fill four or eight of the DDR3 memory slots with quad channel RAM. 64GB was too much, 8GB was too little. I chose 32GB, four 8GB sticks of Corsair Vengeance DDR3 2133MHz RAM ($400) with heatsinks. ECC memory was unnecessary because the CPU doesn’t support it.
Graphics: Nvidia GeForce 660, dual displays, Color Munki calibrator
My rig is for graphics, so the video card would be the place to splurge, right? Surprisingly, advice I heard frequently was: Don’t go overboard on the card. Features that turbocharge gaming cards won’t speed up image editing. An Adobe Photoshop CS6 tech note cautions, “Using more than one video adapter does not enhance Photoshop’s performance. Adobe suggests dual-monitor systems should run off a single card. The card should support OpenGL to offload work from the CPU. That will at least halve the time to run a complex task such as running a filter operation.
I went with the Asus GeForce GTX 660 Ti with 2GB of GDDR5 memory ($300). It draws 150 watts max and calls for a system with a minimum 450-watt power supply, meaning my 750-watt PSU has enough overhead. The card takes two slots but I’ve got slots to burn. Connectors include DisplayPort, HDMI, and two DVI that can easily be adapted to VGA.
The card drives two existing monitors: a 30-inch Dell and an older 22-inch NEC. I love dual monitors and I love 30-inch monitors but twin-30s are about 55 inches sitting side-by-side, too much for the eye to track, so I may upgrade (downgrade) to a pair of matching 27s. If you only get one monitor, give thought to a 30-inch display with 2560×1600 or WQXGA resolution knowing that it will cost about $1500, the same as a pair of 27-inchers.
Color calibration is a must-have for a serious editing system. The xRite Color Munki ($140) color-calibrates the displays, adjusts for changes in room lighting, and tweaks the second monitor to match the color of the main display.
Next page: PSU, storage, and imaging
System unit, power supply unit, and power protection
For the case, I wanted quiet, roomy, and not show-offish. The Corsair Obsidian 650D mid-tower case ($180) fit the bill perfectly with 10 drive bays (four front-accessible), eight slots, room to fit a slightly oversize E-ATX motherboard, integrated USB 2.0/3.0 front jacks, near-infinite cable routing options with protective rubber grommets, three fans, and a case-top SATA 3 port for attaching a bare drive. It had a clear side-panel view port I didn’t need and lacked a card reader.
ExtremeTech’s own Joel Hruska cautioned against overbuying with the power supply unit. With a single graphics adapter, a 1000-watt PSU is overkill. The biggest drain is the graphics card that draws 150 watts max and calls for a 450-watt minimum PSU. I went with a Seasonic Gold 750 watt modular power supply ($100). It comes in a spectacular box with a velvet pouch for the cables, as if this were a single-malt scotch. If it was all the same, I’d rather have plain packaging and pay $5 less.
I have power protection outside the box: a Generac 20kW backup generator (Hurricane Sandy put me on an enforced vacation for 10 days), a Siemens whole-house surge suppressor in the breaker box ($125; clamps down surges coming from outside), and for the new rig, an APC line conditioner (because generator power can be dirtier than utility power) and a UPS for the 10-20 seconds until the generator kicks in. (Yes, I’m a Type A.)
Storage: Dual SSDs, 4TB drives, RAID controller
Dual SSDs are the way to go: one for the operating system, one for data for hard drive caching (or most used files). If (when) something trashes the OS drive, you don’t lose data. Several people cited the high-end Corsair SSDs as worth what they cost. The first time you install an SSD, it blows you away compared to a hard drive, but the second time around you’ll look at the difference among SSDs. I went with a high-end Neutron GTX, 240GB for the boot drive and 480GB for SSD caching.
To store a million photos, I wanted to use four 4TB drives in a single software RAID 5 volume. With RAID 5, available storage equals the capacity of the array minus one drive or 4TB times three, or 12TB. Unfortunately, the place for SSD caching is the four Marvell-chipset 6Gbps ports, which is also the best place for the four RAID drives. That left me the choice of backing down to RAID in the four 3Gbps Intel SATA ports or adding a hardware RAID controller. I went with the LSI MegaRaid 9271-4i four-drive controller ($400).
The 4TB drives are Seagate’s mainstream ST4000DM000 desktop drives. These cost half as much as workstation- or enterprise-class drives that experts say you should use. On a good sale, you can get the bare ST4000DM000 for $140, so I keep a couple spares on hand. Plus, I’m backing everything up to a RAID server. To lose data I’d have to lose two of the four RAID PC drives at the same time I lost three of the eight drives on the server (see below).
Imaging: Three scanners, one scanning service
This is the rig for imaging. It has three scanners attached. My Epson Expression 10000 XL is the standard for graphics professionals because of its rugged design and 11×17 (B-size) capacity. You’d be surprised how many documents you scan won’t fit a standard legal-size scanner without stitching. If you’re scanning photos, you can toss more than a dozen at a time on the bed and Epson’s included software automatically splits them into separate files. It’s $2500 street for the current, slightly upgraded Epson Expression 11000XL. Epson sells refurb scanners for $1500 and eBay has them for about $1000.
For bills, reports, receipts and business cards, I use the Neat Receipts scanner. To organize the PDFs, I use Nuance PaperPort 14 ($100-$200), the best tool for scanning and organizing thousands of documents. The software’s downside is Nuance’s incessant pop-ups reminding you about upgrades and enhancements.
For scanning slides and negatives I use the Nikon Coolscan 5000 film scanner. It’s no longer produced but available on eBay for $1000-$2000, depending on condition and demand. It does scans quicker and better than film attachments on flatbed scanners. Slides scan quickly if you also get the SF-210 slide feeder. Buy the Nikon scanner, use it for a year or two to scan your old negatives, then resell it for the same or more than you paid. If you find the idea of scanning thousands of old images to be overwhelming, send images out to a service. My favorite is ScanCafe, about 25 cents per scan.
Imaging software is Adobe Creative Cloud as a monthly subscription (currently $50 but check for academic and other discounts). Of the bundle, Adobe Lightroom 5 gets the most use. It organizes, geocodes, and tags all my photos, but no face recognition yet. It’s also good for general photo editing. For serious edits, I use Photoshop. For video editing, Adobe Premiere. No surprises there.
Two utilities get a lot of use: Anthropics Portrait Professional ($50-$150) quickly cleans up faces, removing minor blemishes and wrinkles. Yes, this is the software that makes round faces thin. Red Giant PluralEyes ($200) automatically synchronizes the audio from multiple sources, say if you’re using two video cameras, or replacing the camera audio with a separate audio recorder. When I shoot the same scene with a video DSLR and two GoPro Pro Heros, plus an audio track captured on an external recorder, PluralEyes a godsend.
Next page: Peripherals, and backup
Peripherals: keyboard, mouse, card reader
All my peripherals are markedly better than what you get with a manufactured PC. I was torn between two keyboards. I got them both. The wireless Logitech K800 ($75) is more compact and has a softer, quieter touch. The Corsair Vengeance K95 (right, $150) has 18 preset keys beyond the usual function keys and a USB passthrough. It feels great and it must be powerful; it draws power from two USB jacks on the PC. This is a gamer keyboard but it’s great for typing. I switch between them on days when I want a different typing feel. Most Logitech wireless devices using a single unifying USB-port wireless transceiver, so I went a wireless Logitech mouse, headset, and webcam.
I added a Rosewill internal memory card reader ($25). Read carefully when shopping: A combo card reader-USB port device marked USB 3.0 sometimes drives the jacks but not the card slots at USB 3 speeds and it may lack mounting bracket and faceplate if your accessible bays are 5.25 inches. The external Kingston USB reader ($25) runs at USB 3.0, which is mandatory. Connecting your camera directly to the PC generally is a slow USB 2.0 connection.
For wireless, I added a tiny external USB transceiver ($20) and an internal Asus dual-band WiFi adapter ($100). Why? A couple times I year when I need to do intensive photo editing offsite, I’ll take my most powerful PC and the big monitor on the road with me. It’s easier to find wireless than wired.
Backup: Synology 8-drive server, 18TB-plus
I wanted all my files backed on a NAS, or network attached storage, device. The choice was easy, since I’ve owned two Synology four-bay servers that never hiccupped. I went with the Synology DS1813+ with eight bays ($1000 bare) and eight Western Digital Red NAS WD30EFRX 3TB drives ($135 each, $1100). It has four Ethernet jacks and two expansion jacks for five-bay expansion boxes. The DS1813+ can be set up with the usual RAID configurations or (my choice) Synology Hybrid RAID (SHR). Think of it as RAID 5 but with the ability to use different drive sizes and maximize capacity rather than be limited by the smallest drive in the array.
Total capacity with SHR for N drives is N-1 or N-2, depending on whether you set up for one- or two-drive failure tolerance. In a system set to survive the loss of two drives, that’s 18TB based on eight 3TB drives. As you replace or upgrade drives, once two or more higher-capacity drives are installed (4TB instead of 3TB), SHR bumps up the available capacity. With the eight-bay main unit and two five-bay expansion units, all filled with 4TB drives, that could be as much as 72TB of raw capacity, 64TB of actual storage able to survive the loss of two drives. What seems unfillable today may be overloaded in a couple years.
Before you commit to so much data in a single box, no matter how much redundancy, read up on RAID and RAID rebuild times. The mother and father of scare stories said RAID 5 would be dead as of 2009 and RAID 6 is dead as of 2019. It has to do with the possibility of unrecoverable read errors (very small number) compared to the number of bits on the drives (very large number) while you replace a bad disk and rebuild the array. Scary stuff, but then you haven’t heard of thousands of RAID 5 server failures, either. (Yet?) There is also a plenty of discussion on desktop vs. workstation or server class hard drives and their failure rates. A 2007 Google white paper said hard drive failures are hard to predict, even with SMART reporting on drives; it didn’t, unfortunately, talk about brand reliability or desktop vs. workstation drive reliability. At the time of the survey (2006), Google said 2- and 3-year-old drives suffered about an 8% failure rate. That’s why I keep at least one spare drive handy.
To be cautious, I have one of my old Synology NAS boxes with 6TB (4 x 2TB, RAID 5) capacity in an outbuilding connected by underground wired Ethernet. I have two 4TB bare drives that plug into the SATA port on top of the Corsair case (very cool) and back up the PC; it goes in the garage until I go to the safe deposit box. The Blu-ray writer also backs me up, 50GB at a time. So in theory, my new rig is protected — the weak link is making frequent backups that are taken offsite.
The other rig: Xeon system, cheaper not dirt cheap
I had two worn out PCs and built two new ones. The second was with an eye toward affordable price-performance, less than $2000. I went with similar but more affordable components: Xeon four-core E3-1230 3.3-GHz socket 1155 CPU, an Asus Workstation-class socket 1155 workstation motherboard (P8Z77 WS) that also accepts some Core i7, i5 and i3 CPUs; four 8GB sticks of Kingston DDR3 non-ECC memory; a generic GeForce 660 GTX card; two Kingston 240GB HyperX 3K SSDs; a single 4TB Seagate hard drive; a DVD burner, a Corsair Enthusiast 850 PSU; wireless Logitech keyboard and mouse; and a Corsair Carbide 200R mid-tower case. The system cost with Windows 8 is about $1800, without monitor. If the CPU runs warm with the stock fan, I may upgrade the CPU cooler, but so far so good.
What I’d do differently next time
All in, my Core-i7’s rig’s hardware runs $10,000 if you count the loaded PC, the NAS device, the scanners, and power protection other than the generator. The system itself is $4600 with Windows 8 (64-bit), or $4300 excluding external power protection and color calibration. For an apples-to-apples comparison with the Xeon second system, back out the RAID controller, three of the four hard drives, the webcam, and the wireless adapters. It’s down to $3100 vs. the Xeon system’s $1800. Given how much time I spend at my PC each day, I don’t regret going high-end with the CPU, motherboard, RAM, big SSDs, and case.
In hindsight, once I got the Synology server, I’m not sure RAID 5 was necessary inside the PC — especially once it turned out the fastest RAID-capable motherboard SATA ports were taken by the SSD drives and I needed a hardware RAID controller. I could directly attach two or three hard drives as separate volumes, split images across the drives, and do constant backups. But this was an in for a penny, in for a pound build.
4TB drives are now near price parity with 3TB so, for the server, filling six of the eight bays with 4TB drives (two-drive failure protection) is about the same cost, capacity and security as eight 3TB drives. Later I could start filling the empty bays with 6TB drives such as the Western Digital 6TB helium-filled drive released last fall and just coming to market. But that’s all hindsight.
How my rig worked out
After using my new Core-i7 rig I’m satisfied — say 9 on a 10 scale. Photoshop time-sink processes (like blur filters) execute quickly if not in the blink of an eye. What took 25-30 seconds now takes 5-10 seconds. The most spectacular difference is scrolling through a set of a thousand photos in Lightroom. Tap the right arrow button and the next image pops on-screen immediately. That’s Andy Grove’s blink-of-an-eye quick. Nothing I’ve done in Photoshop has ever bogged down, even with dozens of images opened. With so much memory, there’s little concern about what is the right drive for the Adobe scratch disk since it’s unlikely to be called on.
One operation that didn’t improve as much was the Lightroom import and preview rendering of 1,000 18-megapixel JPEG photos. That took 30-45 minutes on older PCs. I got it down to 15-20 minutes which is nice but not dazzling. I still make coffee and wait but don’t have time to go out for coffee. The biggest improvement in the Lightroom import was the file transfer from CF card to disk, 8-9 minutes down to 3 minutes, but that was virtually all the result of the new PCs using USB 3.0.
One of the nicest things about building my own rig is what I didn’t get: crapware. PC makers get spiffed to include marketing offers and software trial versions, none of which most users want. That alone makes me want to build again.
Check out some other ExtremeTech rigs, from Sebastian, Sal, and others.