external hard drives and how do you set up, and save old images and memory card files?

[spudhead]

So I have asked similar question before so lets me ask again for tips and advise please.
So I have 2 external hard drives full of saved memory card files sooc, and saved edited files which are almost full and to be honest a bit of a mess, I also keep on the laptop my latest edits from the previous and current year. I have just bought a new 5tb external and am hoping to save to that most of what is on the other 2 hard drives but a lot more orderly fashion and then clean up the old hard drives. So I have saved so far to the new hard drive sooc memory card files, so any tips on how you guys sort your files and setup your external hard drives. thanks guys

 

[Clix Pix]

I use a lot of external drives for various purposes. Most are external SSDs of various capacities and a few are somewhat older larger-capacity HDDs.

On my primary computer, the one I use for image editing, I keep image files from the past year and current year. When I return from a shooting session I transfer the files to the computer, cull and edit them. Once edited, and renamed, the files go to my Pictures folder (I'm on a Mac) into the folder for the appropriate year and month. Each edited image file is named whatever the subject or event had been.

Sometimes I don't get around to culling and processing image files right away, and when that is the case I drop those files on to an external SSD named RAWs, and when I am ready to work on them, they're ready for me without taking up space on the computer's internal drive. Depending upon the time of year (autumn and spring I shoot a lot, lot more than in winter or summer), I also designate the month as well: "Marach 2024 RAWs"). Usually the files which land in the RAWs external SSD are the large ones, such as when I've spent some time shooting the birds frolicking around on the water or maybe one particular bird standing around on the pier. Thanks to Continuous High + that's set up in my A1 clipping along at 20 fps it doesn't take long to generate 1200 or more images on one 160 GB CFE card.

I have several external SSDs for backups, and I duplicate them so that one set goes to my safe deposit box at my bank once a month for off-site storage. Each month I take the freshly updated set to the bank and bring home the one which has been in there for a month.

One set is devoted to several years' worth of my Pictures Folder.s Another is specifically for Current and past Edited Image folders, arranged again by year and month. If something has been shot during a specific time frame during the month, too, say a week or a day, those folders are also dated and organized within the month. (For example, "October 19, 2023 Concert at City Winery" is filed within the year 2023 and the month of October.)

So, earlier today I did a little shooting, and after culling and processing, the completed edited images went from the desktop to the Pictures Folder into several folders:

2024 - March - Edited Images (this picks up all the images shot within a given month)

Next, 2024 - March - Week 11 (just starting a new week today)

This coming Sunday I will spend some time copying this week's group of edited files to my backup SSDs: two sets for home, a third set for the bank. At the end of March (actually, beginning of April) I will then take the freshly updated set to the bank.

My image files are handled separately from my other files,; however, I actually also use additional external SSDs for backing up my various other folders on the computer: Documents, Movies, and Music. I also have one drive which is for travel, and that includes everything current: Image files, Document files, Music Files, Pictures Folder Files. When traveling, if it is a situation where I'll be taking photos, I also always take along at least one empty external SSD for copying/backing up newly-shot RAW files while on the road.

Once I am done with editing, I don't keep my old RAW files. I used to do so, but eventually I figured out that if I haven't processed something in a year or two, then it's never going to be processed anyway. Those files just take up too much space.

So that's basically how I have things set up now. Over the years, I've adjusted and refined how I do things and as my image files continued to grow and/or as external SSDs became larger in capacity and lower in price, I made changes accordingly. My older and largerHDDs are specifically archival drives only and they contain all the stuff from my earlier years in digital photography along with the folders from more recent years. I haven't gotten around yet to putting 2023's files into those drives but need to get to that one of these days pretty soon.

 

[spudhead]

I use a lot of external drives for various purposes. Most are external SSDs of various capacities and a few are somewhat older larger-capacity HDDs.

On my primary computer, the one I use for image editing, I keep image files from the past year and current year. When I return from a shooting session I transfer the files to the computer, cull and edit them. Once edited, and renamed, the files go to my Pictures folder (I'm on a Mac) into the folder for the appropriate year and month. Each edited image file is named whatever the subject or event had been.

Sometimes I don't get around to culling and processing image files right away, and when that is the case I drop those files on to an external SSD named RAWs, and when I am ready to work on them, they're ready for me without taking up space on the computer's internal drive. Depending upon the time of year (autumn and spring I shoot a lot, lot more than in winter or summer), I also designate the month as well: "Marach 2024 RAWs"). Usually the files which land in the RAWs external SSD are the large ones, such as when I've spent some time shooting the birds frolicking around on the water or maybe one particular bird standing around on the pier. Thanks to Continuous High + that's set up in my A1 clipping along at 20 fps it doesn't take long to generate 1200 or more images on one 160 GB CFE card.

I have several external SSDs for backups, and I duplicate them so that one set goes to my safe deposit box at my bank once a month for off-site storage. Each month I take the freshly updated set to the bank and bring home the one which has been in there for a month.

One set is devoted to several years' worth of my Pictures Folder.s Another is specifically for Current and past Edited Image folders, arranged again by year and month. If something has been shot during a specific time frame during the month, too, say a week or a day, those folders are also dated and organized within the month. (For example, "October 19, 2023 Concert at City Winery" is filed within the year 2023 and the month of October.)

So, earlier today I did a little shooting, and after culling and processing, the completed edited images went from the desktop to the Pictures Folder into several folders:

2024 - March - Edited Images (this picks up all the images shot within a given month)

Next, 2024 - March - Week 11 (just starting a new week today)

This coming Sunday I will spend some time copying this week's group of edited files to my backup SSDs: two sets for home, a third set for the bank. At the end of March (actually, beginning of April) I will then take the freshly updated set to the bank.

My image files are handled separately from my other files,; however, I actually also use additional external SSDs for backing up my various other folders on the computer: Documents, Movies, and Music. I also have one drive which is for travel, and that includes everything current: Image files, Document files, Music Files, Pictures Folder Files. When traveling, if it is a situation where I'll be taking photos, I also always take along at least one empty external SSD for copying/backing up newly-shot RAW files while on the road.

Once I am done with editing, I don't keep my old RAW files. I used to do so, but eventually I figured out that if I haven't processed something in a year or two, then it's never going to be processed anyway. Those files just take up too much space.

So that's basically how I have things set up now. Over the years, I've adjusted and refined how I do things and as my image files continued to grow and/or as external SSDs became larger in capacity and lower in price, I made changes accordingly. My older and largerHDDs are specifically archival drives only and they contain all the stuff from my earlier years in digital photography along with the folders from more recent years. I haven't gotten around yet to putting 2023's files into those drives but need to get to that one of these days pretty soon.

Thanks for info and tips I should have said I have 2 ssds and one hard drive.

 

[Roland]

No filing system. Just folders with the year and month. Special trips get there own folder. When I need a picture, I just search. Takes less time than an very complicated and time consuming filing system.
I have a 12tb main external harddrive with all the pictures I took with various cameras over the decades. I fill up now 2 5tb external harddrives as backups and store them alternating off site. When they are full, I buy 2 new ones. I have no pictures or data files on my computer at all.

 

[Richard A]

I don't use any spinning hard drives for storage any more. All SSD's. I do still use hard drives for backup though. I have one set permanently attached to the computer & another two sets that I rotate off sight. This is the SSD enclosure I use: OWC Express 4M2

I have an Apple Mac Studio computer. As for how to arrange all your files - I think this is a personal thing. I just have mine filed under the camera, year, RAW, Jpeg or Movie file. I also have a folder for each camera & year of completed edited files & I have a work in progress file. This works for me but maybe not for anyone else. I use Adobe Bridge for browsing files.

 

[Tim Mayo]

I just have yearly folders and then within them I name the folders as a specific event / activity together with the month. It just makes it easy for me to find things rather than also creating individual monthly folders. I also have folders for my favourite bird, family shots etc... That said, as much as I try to be organised I still end up with files all over the place and in random folders!

I'm using a Macbook and currently have a 6TB Cloud subscription to deal with backups. My favourite shots and family shots also get backed up onto a Samsung T7 SSD. In the past I also used a Synology NAS box in a RAID 1 config but I had quite a few disk failures so stopped using it.

 

[spudhead]

Since we're on this subject, has anyone ever seen an external drive or a docking station that holds a drive or two, and also has a card slot or USB for direct transfer? I saw something once that suggested it would do that. I'd like to be able to go from the card to the drive and not even involve the computer until I'm ready to process.

No Tim but that is what I would like to do to be honest, just store memory card files sooc and access when I want to edit

 

[Richard A]

Since we're on this subject, has anyone ever seen an external drive or a docking station that holds a drive or two, and also has a card slot or USB for direct transfer?

There used to be similar devices available for transferring files when travelling. I've never used one or seen anything like that for a while.

 

[spudhead]

I think Western Digital made a portable drive that had an SD card in it, but you're correct it's no longer available.

I found this after a couple of failed search terms, could be just the ticket.

https://www.amazon.com/RSHTECH-External-Duplicator-Tool-Free-Supported/dp/B07WR9N3KW/ref=sr_1_4?crid=U0FRFPK3ACHN&dib=eyJ2IjoiMSJ9.8wehO7wZtmzi0n9wesji9uO2TCPIpJLewzBGXvoJ1TQxBeKu5TCcti6EMWpUoHvHXzJdOPd5hWYyIAUx-Wea63EjX8pZ8jHR-SE1R_ABMfFoFKx_XDFdIp29yOTRFgayGWhj7l1qzb8-ynemlTRruHGk6mcv_S55d4g99V7bSNxBs8_j5ozPHG9I8wY4uez8pK67WWZCwlatUrpU7I1Y7JEK2embbIzYEbyE8m6Hw1k.G5To1hUF-qKLJJKWWHGAr-WQaxV8JPiRUuOpcubpktI&dib_tag=se&keywords=External+SSD+docking+station+with+built+in+card+reader&qid=1710246335&sprefix=external+ssd+docking+station+with+built+in+card+reader,aps,84&sr=8-4

very interesting I have seen images and reviews of the same with plug in ssd slots

 

[spudhead]

Another idea. Has anyone ever tried just transferring between two USB ports on a computer? Example:

Card reader in USB port 1
Portable SSD or HD in USB port 2

Open both drives, drag and drop.

Seems to me this could do a direct transfer from card to drive via the computer without getting the internal drive involved. Would it transfer as quickly as the USB ports allow, or at least a lot faster than going to my internal HD?

Unless someone shouts me down, I am going to give this a try tonight.

doing it as we speak but 40 64 cards slow work mate

 

[spudhead]

doing it as we speak but 40 64gb cards slow work mate

 

[spudhead]

What kind of transfer speed are you getting? When uploading to my internal HD, it seems to flatten out around 50MBps. If I could even get to 150-200 MBps I'd be thrilled.

less than that, but I need to do this so I have a backup of sooc images so when it is done I will just add the next card

 

[Richard A]

Another idea. Has anyone ever tried just transferring between two USB ports on a computer? Example:

Card reader in USB port 1
Portable SSD or HD in USB port 2

Open both drives, drag and drop.

Seems to me this could do a direct transfer from card to drive via the computer without getting the internal drive involved. Would it transfer as quickly as the USB ports allow, or at least a lot faster than going to my internal HD?

Unless someone shouts me down, I am going to give this a try tonight.

Of course you can do that. Speeds can vary widely depending on card speed, port speed, drive speed, card reader speed, cable speed...

I don't have any photo or video files at all on the internal system drive on my computer.

 

[spudhead]

I figured it would work, just never tried. Now, the question is how fast it is on my computer.

My goal would be to upload files from cards to the external SSD and use that as their temporary home while sorting/culling/processing, then save the processed jpegs and corresponding RAW files to the internal HD. Once done, I'd erase the temporary SSD so it's ready for the next time.

I do keep some images on my computer. Once in a while I back it up to two external drives then delete all but a handful of folders that I may need to access easily. I have an external HD for storage that goes in the safe and an external SSD that stays on the desk. By culling and deleting unwanted images I am able to keep the volume fairly low, even when I come home with 2,000+ images from a day at the track.

it is working just fine I have always done this but things have got messed up so time to sort it and keep this ssd just for sooc files only

 

[Thad E Ginathom]

I do still use hard drives for backup though. I have one set permanently attached to the computer

I strongly advise against permanently attached.

Consider that one way that data can be lost is by electrical spikes. If a spike hits your computer, chances are it will take out anything attached to it as well. Any copy that is on, or attached to , the same computer is not a backup, it's just another copy.

Another idea. Has anyone ever tried just transferring between two USB ports on a computer? Example:

Covered already by others. Yes, often. I have even reformatted an external drive and copied the entire contents back off the other backup drive.

 

[spudhead]

Well that was tremendously successful! I put a Sony Tough SD card with 400 images from the A7 IV in the reader and connected it to one USB port, and transferred them directly to a 2TB Samsung SSD in another port. It went up to 210MBps and stayed there. Took less than 2 minutes.

Going to buy another SSD to use as a temp storage so I don't have to risk one of my permanent storage drives. Maybe not as fast as some, but 210 beats 50 +/- all day!

So how were you loading files for storage before?

 

[spudhead]

Card reader to the internal hard drive. The problem is the hard drive will only write up to 50MBps or so. I have searched and searched trying to figure out what is running in the background that's taking up the HD's resources. Have done SpyBot search and destroy, disk cleaners, etc. Can't find anything.

This takes the internal out of the equation. I can upload to the external SSD, process them and save the jpegs to the HD. When I come home from the track with over 2000 images on the A1, the difference in time is going to be substantial.

Ok all good then

 

[Richard A]

I build my own external SSD drives.
Brownie, what computer are you using with that slow drive in?
Thad E Ginathom, that’s why I have another two sets of backups that I rotate off site…

 

[Clix Pix]

Another idea. Has anyone ever tried just transferring between two USB ports on a computer? Example:

Card reader in USB port 1
Portable SSD or HD in USB port 2

Open both drives, drag and drop.

Seems to me this could do a direct transfer from card to drive via the computer without getting the internal drive involved. Would it transfer as quickly as the USB ports allow, or at least a lot faster than going to my internal HD?

Unless someone shouts me down, I am going to give this a try tonight.

Sure! I do this frequently, plugging in two external SSDs and transferring data (usually image files) from one to the other.....no big deal there, is there?

 

[spudhead]

Found a Samsun 500G portable SSD on closeout for $49. This'll be perfect for my use.

Just paid £106.00 for the 5tb ssd wd better prices if you are a member Tim free postage here as well

 

[Richard A]

The reason the transfer of files slows down is because the cache fills up. Like the buffer in your camera. That’s why I build my own.

BTW, having a drive dedicated to that task is a good idea I think.

 

[AlphaWorld]

Since we're on this subject, has anyone ever seen an external drive or a docking station that holds a drive or two, and also has a card slot or USB for direct transfer? I saw something once that suggested it would do that. I'd like to be able to go from the card to the drive and not even involve the computer until I'm ready to process.

Not an external drive, but I think there are some NAS boxes which will do that. Trying to remember which brand it was, and I am drawing a blank. Sorry about that.

Of course, with NAS boxes is you'll get some data redundancy.

 

[Richard A]

The cache in the SSD...

Not all SSD's are equal.

 

[Thad E Ginathom]

<<Only for Linux folk>>
Although others could use the same methodology

Copying from camera...

I copy pics from camera to computer via a staging location. I use a gui app called grsync (Linux) but it is a front end to rsync and it tells me that this would be the command line:

rsync -r -t -v --progress --ignore-existing -s /media/[me]/C816-666C/DCIM/ /Library/pictures/0-a7iv/DCIM

I have to change the "C816-666C" when I format the sd card, as it gets a new label.

As you can guess, it is a sync process rather than just a copy. It won't copy files that have been copied already.

I manually copy the new directories from that location to my pictures/year/month folders. I keep the camera-allotted folder names, just appending the event name. "Create New Folder" on the camera covers multiple "events" per day. (so long as I remember to do it!)

Copying to backup disk...

Here I use my own scripts, also using the rsync command. My script looks for a connected drive and puts it in a variable. But basically does...

rsync --delete -r -t -h -p -o -g -x --progress -l -H -s --exclude=.Trash-* <current year directory> <Backup-Drive>/path/to/same

I do this whenever I have done some adding, culling or editing. The delete option replicates deletion on the backup.

No, I don't remember what all the letter options do. I worked all this out and scripted it several years ago.

 

[AlphaWorld]

Another idea. Has anyone ever tried just transferring between two USB ports on a computer? Example:

Card reader in USB port 1
Portable SSD or HD in USB port 2

Open both drives, drag and drop.

Seems to me this could do a direct transfer from card to drive via the computer without getting the internal drive involved. Would it transfer as quickly as the USB ports allow, or at least a lot faster than going to my internal HD?

Unless someone shouts me down, I am going to give this a try tonight.

Yes, that works. It’s still going via the computer (my first read of what you wrote was that you wanted it to transfer without the computer - blame too many long days and an excess of blood in my caffeine stream), but the data skips the slow internal drive in the computer - card to RAM to external drive.

 

[Thad E Ginathom]

It’s still going via the computer

One of my brain cells says that SCSI could transfer direct between two attached devices. Is it right? It's a long time since I had anything to do with it.

Anyway, I don't think that SCSI is a viable alternative these days. At least not for home use.

 

[AlphaWorld]

One of my brain cells says that SCSI could transfer direct between two attached devices. Is it right? It's a long time since I had anything to do with it.

Anyway, I don't think that SCSI is a viable alternative these days. At least not for home use.

SCSI did have the ability to do that (device to device commands), but you won't get that by dragging and dropping files on the GUI.

I think that may have been limited to old-school parallel SCSI (I used both "narrow" (50 wire) and "wide" (68 wire) SCSI - it always amused me that the narrow cables were wider than the wide). I don't know if it still has that ability when we start talking about SAS (Serial Attached SCSI). Back when I was assembling my own computers I built systems using ESDI, SCSI, and eventually IDE/PATA. It went almost entirely SATA for a while (SAS in enterprise), but I haven't built a machine in almost a decade, and I suspect that these days NVMe has probably taken a lot of the share.

 

[AlphaWorld]

I was looking at two options, one an external drive(s) with a docking station that has card readers/USB ports built-in and would transfer sans computer, and the other using the computer's USB ports to transfer from card reader to external SSD without getting the computer's internal HD involved. I tested and opted for the latter, it works better than anticipated.

The specialised boxes that can do the copying for you tend to be a lot less flexible. Attaching the external SSD - so much more flexible (and it's exactly what I do!)