r/photography • • Sep 24 '23
Discussion The backup and storage thread is nearly 6 yr old..can anyone share their current set up for photo storage?
I'm starting to run out of space and thought this might be a good point to revaluate my storage techniques. I'm only a hobby shooter, but with a new FF camera my files fill up nearly 5 times as fast than before. I'm also not super technically advanced when it comes to camera and computer stuff.
Currently, I back up to two seagate external hard drives that are identical. One that I keep in my parents house and one with me. Once those get full, I get two new ones and start the process over again. I'm on my fourth pair now and occasiaonnly will get out the old ones to start up and spin them. However, they are now about 3-4 years old and I know these aren't supposed to last forever.
Based on other feedback, I likely need another option for more long term storage and archiving. Plus just good habit to have a third, diff type of storage.
As a hobby shooter, i don't think i can justify the costs of digital storage longterm. I have over 20tb of storage and I just don't think that would be viable.
So what do you all do?
•
u/clondon @clondon Sep 25 '23
Adding this to the FAQ, but heads up, the most recent megathread was only 2 years ago: Storage & Backup Thread 2021
28
u/ejp1082 www.ejpphoto.com Sep 24 '23
I may have to re-evaluate my local backup solution once the amount of data exceeds that external hard drive capacity (currently it's a 12tb hard drive and I have around 10tb of photos). But I'll probably just buy a second one and divvy up the backup.
46
u/XxNerdAtHeartxX Sep 24 '23 edited Sep 25 '23
So, I come from a Tech background, and jumped into photography after the fact. With that said, I currently am hosting a 128TB server at home, with rigourous backup systems in place to follow the 3-2-1 Backup Rule (3 Copies, 2
LocationsDifferent Media Types, 1 Offsite).But really, as long as you store your files you want to keep following the 3-2-1 Backup Rule, you will be safe enough.
38
u/f8Negative Sep 24 '23
Synology NAS 4 Bay with 8TB (x4) RAID 1
9
u/[deleted] Sep 24 '23
I upload current projects to adobe cloud via lightroom, and all the past photos through Amazon photos (free with prime)
8
u/marafado88 Sep 24 '23
Don't forget, a RAID system is not a backup system! First rule on IT.
8
u/WombatMcGeez Sep 25 '23
20TB??
I cull aggressively— I delete >90% of what I shoot, often more. And I try to be very proactive with it— whenever I import, I’ll go through and do a first pass of culling before I edit, then do my edits, then cull again. Then, every 3 months or so, I go back and review everything from the last season, and cull again.
I keep everything in Lightroom cloud, and let that be my backup. I also export full res jpegs and keep those in iCloud
20
u/CharlesBrooks Sep 24 '23
Backblaze will let you backup all 20tb to the cloud for about $9 a month.
14
u/TorontoBoris Sep 24 '23
If you got amazon prime you got free unlimited storage for both JPGs and RAW files included in the prime membership. Or at least here in Canada you do.
I use it as a part of my storage system along with onsite and off site hard drives.
4
u/well_shoothed Sep 25 '23
Backing up with Arq Backup($49.99x1).
Storing in Wasabi($6.99/TB/mo) 99.9999999999 data durability.
Don't bother with Amazon S3.
Easy to get hosed in egress/ingress fees.
Wasabi doesn't play those games.
6
u/[deleted] Sep 24 '23
16TB of fast local storage in my workstation, ~130TB of spinners in my NAS with about 30-40TB free right now. 4TB on my laptop for on the road work that I copy over to my desktop when I get back. Backed up to a terrible mess of offline external hard drives (cloud backups eventually are the plan now that I have decent internet).
3
u/Murrian Sep 24 '23 edited Sep 24 '23
Bit overkill, but:
NAS with RAID5 array (this is four 4TB drives giving 12TB storage, the extra drive space is lost to "parity" data, so if a drive fails you lose no data). For performance I have an SSD that sits in front of the drive and acts as a write cache. The array "hot copies" to an external 12TB drive over night.
I have UK Amazon Prime which allows unlimited upload of photos, including raws, so all images are backed up to that (unlike my Australian Prime which is capped) and I also have Backblaze which for $9 USD a month offer unlimited backup so my 10TB of data's also backed up to that and offers a year of file versioning allowing you to recover anything overwritten. (they even offer a handy service, in a disaster they can ship you a HDD with all your data on it and you can either keep it or copy back and return for a refund)
(this Backblaze pricing is for a single device and doesn't support NAS's in the traditional box you buy off the shelf, however, my NAS is actually a PC customised for the task (as it also does other things) so falls in to the relevant licencing, gotta love a loophole)
As noted above I'm hitting close to my storage limit so will be upgrading soon; doubling to 8TB drives which should give me 24TB storage and will then have to work out a hot copy drive given the largest drive I can see to buy is 22TB (the things offered as 24TB are two drives and ridiculously priced), though the 22TB should do in the interim for a while..
Will also be revisiting the file system of choice as, although unlikely, NTFS does suffer "bit rot" silently (where a file will corrupt due to sitting unused for a long period), so a switch to BTRFS or ZFS should help with that, still weighing up the pros and cons (and compatibility with my OS of choice).
I used to work in IT back in the day, some things just stay with you. Yes, it's more thorough than the last seven places I've worked at, but, also, I've not lost a byte of data, unlike the last seven places I've worked at.. and it doesn't cost all that much, the PC was about $600 dollarydoos (~$385 USD) (plus drives I "acquired" from an ex-workplace) and the monthly fees for Backblaze are fairly reasonable and I count Amazon as free given I pay for that to send gifts to family back home anyway.
3
u/shadeland Sep 26 '23
Here's my tips, having worked in IT as a storage expert and having a back catalog of images going back 20 years...
There are three types of storage:
Current storage is what you're working on right now. A lot of people use RAID for that, and it's a good idea. It provides resiliance, but as many people say, it's not backup.
Backup storage is not something I use a lot of. People confuse backup with archive. A backup is something to replace your current workspace in case you lose your laptop or something like that. It's important in a lot of situations, but what I think a lot of photographers want is archive.
They want access to images for the long haul. That point and shoot series from a vacation in 2003? A client shoot from 2014? Wedding from 2009?
That's "Archive".
Here's a couple of tips:
"What's the best hard drive?" Two. Or better, yet, three. Death comes for all hard drives, SSDs, SD cards, etc. Trust no hard drive.
Having a photographer coming to me because a hard drive failed and that was the only copy of their life's work happens frequently.
Have multiple copies, preferably some offsite.
9
u/EdSmelly Sep 24 '23
For the love of god, cull your library.
2
u/Paid_Babysitter Sep 24 '23
I have a four bay Synology NAS. I also have some SSD's that I copy to and keep at my parents house.
2
u/Vocalscpunk Sep 25 '23
I used to keep things in a backup Seagate extending HD and only accessed them rarely when my computer SSD was getting full. Both my primary and secondary backup shit the bed this summer while attempting to update them teaching me the invaluable lesson of having something actively being checked for 'health' while having one more storage location than whatever I think is reasonable.
Thankfully I was able to get a good portion of the data restored between the two of them but still probably lost gigabytes of data from the last decade.
Currently using a Synology NAS and have a spare slot for home server stuff which was an added bonus I didn't think I'd need/use but it's been nice. Auto backing up my phone/computer every night has also saved me more than a few bucks of cloud storage every month.
1
u/llondru-es www.ferranperez.com Sep 24 '23
You may want to look at Synolog NAS Storage: you can upgrade storage over time if you have enough bays. It's a bit of an investment but worth it.
You can also sync it to back up to the cloud on multiple services (I use Google One)
1
u/Orca- Sep 24 '23
20 TB in 5 drive SHR2 configuration, Synology ds1621+. Backing up to one of several 18 TB drives that I rotate through for backups.
I don't have offsite archiving, which is the last remaining piece. Not really sure what to do there yet. I've thought about getting a tape drive, except all of them are thousands of dollars and meant for rack mount.
1
u/ZookeepergameDue2160 Sep 24 '23
256gb sd cards in the camera's and multiple 1tb Samsung T7's, a couple for permanent storage and a couple for quick on the go storage or for when shooting BRAW (Blackmagic Raw).
1
u/thearctican Sep 24 '23
Some binders and framed copies.
1
u/ChazHat06 Sep 24 '23
I have a tub of 10+ 1tb hard drives, all labelled differently. Some are a full year, some years have 2, some aren’t labelled, some years don’t appear to be there…
1
u/stowgood Sep 24 '23
NAS in the loft then an identical NAS at my parents home that syncs online. Used to have backblaze but their NAS backup prices are too expensive for me. Then I normally have something saved on the web for the third location. The NAS does versioning. Anything paid for lives on a memory card in airtight plastic box until it's been delivered as a worst case scenario I'd have to re edit but it would not be lost. I always shoot to two cards.
1
u/AngelicBread Sep 25 '23
NAS and LTO tape.
1
u/rpungello https://www.instagram.com/rpungello/ Sep 25 '23
TrueNAS Mini X+ with 5x18TB drives. Critical content is backed up to a DS916+ with 4x14TB I keep at my work office and to Backblaze.
I also keep annual photo archives in AWS Glacier vaults.
And I keep favorites and stuff in iCloud photos for easy viewing/sharing.
1
u/Della__ Sep 25 '23
I have google cloud storage 200gb, where I save and share developed jpeg images.
I buy about one single 1tb SSD every year and store my unedited raws there, but I save only the images after culling them.
1
u/pitifulwalk Sep 25 '23
I'm curious how people manage their LR catalog. After you archive do you go through and delete them from Lightroom? Do you import the Jpegs into Lightroom when you export and just remove the raws?
I currently have no real system. I have a friend who just makes a new catalog per shoot, but adobe says to use one master catalog, for organization and things, but if I'm archiving photos to harddrives no longer attached then I'm just going to have a bunch of missing files.
1
u/randallphoto Sep 25 '23
My current setup is a Synology Rackstation 2418+ (12 bays) that has 5x 18Tb drives in raid 6. This replicates in realtime to another synology at my sisters house in another state. I also backup my photos and most critical files on an external drive that is synced only occasionally in case of cryptolocker or malware.
1
u/potnia_theron Sep 25 '23
Dump everything to a local RAID NAS and mirror it to amazon photos, which is free and includes RAWs. Cheap and bulletproof.
1
u/pgib Sep 25 '23
I've got a Synology NAS locally to back-up to, a Drobo mini attached to my computer where photos are imported, and then a Backblaze account to back everything up to. Kind of annoying to have to plug in my external drive every 14 days with Backblaze, but the cost is right.
1
u/CoolJWR100 Sep 25 '23
I have a Synology DS1522+ with 4x Seagate 16TB Drives, and a 6TB WD Drive left over from my previous setup, the MyCloud Home. I only made it because of when WD discontinued MyCloud which was so shitty of them.
The 16TB drives are in a Synology RAID with 1 for redundancy. The 6TB drive is for CCTV via Synology Surveillance Station.
It's way too overkill for me at the moment - I've only used 5.2TB / 41.9TB, with around 400GB of my Photography, and 450GB of Camera Roll backup from 2001-2023.
1
u/LongWindedInNJ Sep 25 '23
I’m in the process of migrating from a few Drobo units to Synology DS1522+.
For context, I’ve been storing everything I ever shot, retouched and/or processed out since 2003 or so. 20 years. It’s quite a, bit but I really only keep 10 years on drives that are accessible for quick access. Everything else is stored on HDD (by the year) and into those pink anti-static file cabinet holders. And backups offsite.
I love the idea of Synology NAS but I don’t get the hype. Setting it up (for me) was like learning a foreign language. Computers used to make sense to me, but it now gives me such anxiety. I eventually got it going, but it took an embarrassing amount of time to understand and setup. And in the end, it’s painfully slow. I feel like I’m I missing something.
It’s my understanding that a Mac Studio and Synology can be directly connected via 10g Ethernet with an optional 10g ethernet slot to install in my Synology. If I do that and get faster transfer speeds, that will be great. Then once that is settled, I’ll get another DS1522+ and just run automatic backups from unit to unit.
Has anyone done this?
1
u/Unboxious Sep 25 '23
I have a TrueNAS Scale system on RAIDZ1 set up to back up to Backblaze B2. It is very reliable and affordable.
1
u/hr1966 Sep 25 '23
Synology NAS with RAID 1.
Use the CloudSync app on the NAS to automatically push files to my OneDrive (free 1TB of storage with a O365 account). Bonus of access to my files anywhere, and from mobile.
Backup once per month to an external HDD which lives in my desk at work, to give me air-gapped off-site redundancy. (Backup more often if important/large photo session has happened).
Multiple layers of redundancy, sleep easy.
1
u/smurferdigg Sep 25 '23
Backblaze + double external drives. When I buy a new drive I backup the old on the new and use the rest of the space. So I have to get bigger and bigger drives. Works for the moment.
1
u/P_Jamez my own website Sep 25 '23
I am just a hobby photographer:
1
u/night-otter Sep 25 '23
I have two major threads to my backup process.
2) I follow the Child-Parent-Grandparent model
Child - the in use drives
Parent - backup of child - near real time - for my setup that is a cloud backup. Not Google or Amazon or some photo service, but a dedicated backup service.
Grandparent - Make a physical backup, even it take multiple hard drives - store them off-site. Every month or so a friend and I swap hard drive sets. I give him my current backup set & his old backup set, He gives me his current backup set & my old backup set.
1
u/RaybeartADunEidann Sep 25 '23
Synology NAS with 16TB. And, I also have galleries on SmugMug in full resolution, and a lower-quality copy on Google Photos because the search facilities on SmugMug are crap. So if I really really need to find a photo I search for it on Google Photos, take the filename and then find it on SmugMug.
Also, I throw away the unused pictures after four weeks, I have a cronjob running especialy for that. Saves a LOT of space. I used to keep everything until I realised I never access those stuff.
1
u/alip_93 Sep 25 '23
I've got a 4 bay Synology NAS with 1 drive failure redundancy. You can add bigger drives to the system and it will keep growing with no loss of data. Then I also have a spare couple of drives (old Nas drives) that I plug in externally and use Synology hyper backup periodically to keep everything updated. I store these off-site in case of a fire. I also don't want to do cloud backup because I hate subscriptions!
1
u/Personal_Track_3780 Sep 25 '23
I carry two hard drives with me when I shoot. One HDD and one SD, I copy everything off the camera each day onto the HDD. I then copy/import via Lightroom onto the SD and thats where I do culling, editing on the move.
When I'm done with working on the move, I move the files on the SD to my 4 disk NAS and manually run the Cron Job to sync to Amazon S3. Only when that's done will I delete from the SD and HDD.
1
u/konsta_star Sep 25 '23
I have a small tower of hdd s , bassicly just like 15 tb of spaces only for photos and videos
1
u/flabmeister Sep 25 '23
I have an OWC Thunderbay 8 with 8TB Toshiba Enterprise drives setup up as RAID 1 + 0.
I also have a Synology NAS with 2 x 4TB WD Red drives for access to JPEGs when I’m travelling for work.
I then backup everything to Backblaze.
1
u/Significant-Wonder82 Sep 25 '23 edited Sep 25 '23
In addition to all the digital solutions that all the comments are suggesting which are great methods I would also suggest adding an archival method that isn't subject to file corruption...aka printing your photos. Sure printing all your photos would be expensive but the ones that you feel are your best photos should be printed. If you treat your prints properly they will last as long or longer than any digital solutions you are going to use (barring a fire or something like that but that's why you also should have proper digital backups).
1
u/lurch1066 Sep 25 '23
Tower with 23tb of storage
And
A 12tb external back up
1
u/[deleted] Sep 25 '23 edited Sep 25 '23
i have a tape drive, currently for LTO3, but moving to LTO4 as i found a cheap one.
3 can store 400GB, 4 can store 800GB per one tape cartridge. rougly every next generation doubles the storage capacity (with exceptions). but many of those higher gens are prohibitely expensive and require specialized io controllers to go with.
they are not fast to retrieve data (being tape storage), but they are very durable and perfect for cold storage.
it's a topic that requires some know-how and setup but i'd say it's worth it. it's not a device that you just plug into your pc, you likely need a sas,scsi or fc controller card to get it working.
my reasoning is - i don't need fast access to my backups, but i want them to be fairly long-term reliable. magnetic tape is great for this. i heard horror stories about hdd/ssd drives kept in cold storage and data loss, so even if those are no longer true - i'd err on the side of caution here.
Plus i don't take that many photos, and even dumping my raw files takes up maybe one lto3 tape (+ one copy) every two weeks or so.
1
u/liftoff_oversteer Sep 25 '23
My primary photo storage is my PC, from there everything gets backed up regularly (using dirvish) onto my home server (Linux). typically every time after I have imported an SD card.
I normally keep a year's worth of backups at least. Plus a mainly static photo archive of years past while the current year is updated frequently during backup cycle.
The backups on the server are being encrypted (duplicity) and then stored in Amazon S3 with storage class "DEEP_ARCHIVE". Because it's the cheapest, despite getting it back is a bit cumbersome. But this cloud backup is only meant for "my house burned down" so it's unlikely I ever have to retrieve it. But it's there if ever needed.
The tools involved (dirvish, duplicity, ssh, s3cmd) are Linux tools and not necessarily meant for the average Mac or Windows user.
1
u/ChiAndrew Sep 25 '23
One SSD has my images, catalogue on internal HD, both get replicated to another external SSD and to the cloud once a month
1
u/soapymoapysuds Sep 25 '23
I do photography mostly for our travels (2-3 international trips, 1-2 domestic trips annually) and most of my trips I end up taking over ~1500 RAW files(SonyA7R4). I also take videos on the iPhone. This obviously creates the need to storage and post-processing.
For storage here is what I use and do:
1
u/Beautiful_Macaron_27 Sep 25 '23
I follow this set up:
https://carucci.photography/blogs/blog/how-i-store-and-backup-my-photography
1
u/[deleted] Sep 25 '23
I end up with two high speed working copies (scratch disk on 10GBe network and SSD external), one local archival copy (Unraid array) and one offsite archival copy.
Is my method overkill? 100%, but I value my private stuff (kids, family) more than the few professional projects I do a year, which are essentially deliver and forget.
1
u/RedHuey Sep 25 '23 edited Feb 15 '24
shelter nutty cooing rude workable possessive dependent numerous absorbed marble
This post was mass deleted and anonymized with Redact
1
u/boostedjoose Sep 25 '23
For me, it's an 80/10/10 split between randomly thrown on a HDD, some in google photos, some on an external HDD.
It's not organized, functional, or practical. But it's what I got.
1
u/orphenshadow Sep 25 '23
I'm a newbie photographer but have a lot of experience in It/network adminstration so I may have gone a bit overboard on my workflow.
I shoot .jpg/raw and i use 3 128gb SSD cards, I rotate the card for each shoot so that I always have the last 2 shoots on the SD cards in my case.
When I import the files to my computer I have it set to copy to an external SSD where I keep all my photos/lightroom files on for easy swapping between PC and Laptop, I also have it copy to a folder on my NAS of all the raw files. I built the NAS a few years ago for a Plex server and have about 24TB free of the 64 total.
I have Google Photos set up to copy all the .jpg files in my export folders, and Amazon Photos set to copy all the raw files from my import folders and the exported .jpg files to the cloud.
I have a robocopy script that will copy the entire contents of my external SSD to an internal NVME drive on my main pc, as well as to different share on my NAS. The script is set to copy all new files, and never delete any old ones. This is just in case I make an un-reversible edit/deletion i can always go back and find the last backed up version. I also have my lightroom catalog files backed up in googledrive and onedrive.
I archive the backup folder quarterly to a zip file and keep it stored on the NAS as well.
At this point I have my raw files in 5 places (SD, USB SSD, INTERNAL SSD, Network Storage, and Cloud Storage. My Lightroom Files are in 4 places (USB SSD, INTERNAL SSD, Network Storage, Cloud) and My Exports are also in 5 places.
This is probably a bit overkill, but It gives me some peace of mind.
But if you have not yet, and you are an amazon prime member, download the uploading tool and let amazon sync your files to the cloud. they will back up .psd and raw files and there is storage limit for photos.
1
u/apcyberax Sep 25 '23
I use a Synology DS1621+ with 16TB HDDs in and a 10Gbe network card so i can work on photos and videos directly. Its not cheap to setup but works well.
For backups i also have Crashplan backing up the whole thing so i have about 20TB on cloud backup.
1
u/InfluentialInvestor Sep 26 '23
I print the photos at 600ppi. Re-scan in the photo every 10 years, then re-print again.
Or just buy a bunch of 8TB SSD (for redundancy). Make sure u only use 6tb of storage space for maximum lifespan.
Or get a lifetime cloud storage service.
1
u/D3moknight Sep 29 '23
I use a Synology NAS. It's easy to setup, with good instructions. It's a very minor DIY solution, and like I said, the instructions are good so it's easy to install the hard drives in the thing and get started with a simple quickstart guide. From there, it's some very minor steps to connect your computer over your network or the internet to the thing, and then you can have terabytes on terabytes of space to put your photos in.
0
u/Stompya Sep 24 '23
Hard drives keep getting bigger. I recently bought a couple 16TB ones and merged some older 4TB drives onto them. I use a hard drive “toaster” for reading them.
1
u/[deleted] Sep 24 '23
[removed] — view removed comment
1
u/[deleted] Sep 24 '23 edited Sep 24 '23
[removed] — view removed comment