I can't really tell you what to use, but from my personal experience - stay away from glusterfs and drbd. both have caused me serious trouble when trying to run them in a production setup. ceph seems to be pretty solid, though.
If you're on linux or bsd, look into ZFS. Insanely easy to set up and admin, fs-level volume management, compression and encryption, levels of RAID if you want them, and recently they even added phe option to expand your data pools with new drives. All of that completely userspace, without having to fiddle with expensive RAID cards or motherboard firmware.
Huh? ZFS is not 100% userspace. You’re right that ZFS doesn’t need hardware RAID (in fact, it’s incompatible), but the standard OpenZFS implementation (unless you’re referring to the experimental FUSE-based one) does use kernelspace on both FreeBSD and Linux.
Update: I found a partial solution using Nitter and the Link Gopher Firefox extension.
Using a public Nitter instance, one can visit an account page, jump to the bottom, click "Load more", and use Link Gopher's "Extract Links by Filter" tool to extract all links of the form https://nitter.freedit.eu/SpaceX/status/. This method allows one to collect 20 links at a time. It's still a bit more work than I would ideally like, but better than trying to find and copy each tweet URL manually. Also not sure how much longer this method will work given that public Nitter instances will likely stop working soon.
I have the impression that they're recommending this drive just because of the affiliate link money.
They link to a review that they made SIX years ago and at that time it was the fastest drive. Six years it's a geologic era for this kind of stuff. That review is outdated and invalid if compared to modern hardware.
IMHO it doesn't make sense in 2024 to buy a server PCIe 3.0 drive at enterprise prices when you can get a much faster seagate ironwolf pcie 4.0 nvme drive for half the price and in 2TB configuration.
I don't really feel like I'm much the wiser, having read this, on how exactly this works. It's storing data in 3 dimensions in layers and uses 2 lasers in both write and the read process. Why multiple layers in 3 dimensions over a single layer as in traditional optical media would yield better storage density is intuitive but the way they're able to do this is not that well explained. I don't understand the relationship between having 2 lasers and being able to store data in many layers. The fact that one laser disables the effect of the other both in read and in write is confusing, one would think "switching off" the writing process done by... not writing anymore, rather than having a second laser which somehow disables the first but in any case the effect of this is said to allow "spots" (are they like pits?) smaller than the wavelength of the light used to create them which is presumably very small and again makes intuitive sense as to how that would allow increased density and thus storage capacity but doesn't help explain the 3 dimensionality. Also, how does firing a laser at a material presumably burn it away to produce a "spot" (pit?) but firing a second laser at it stops this from happening? Similarly, with reading, how does firing a laser at a spot cause it to fluoresce, yet firing a second laser at it somehow causes it to stop doing that? How bizarre.
On an even more basic level, how do layers work? How does the outer most layer of the readable surface of the disc not block or interfere with the ability to read or write the next layer beneath it and so on?
The spot size is the size of the point the laser focuses on. It seems with the material they've used that there is some kind of interference between the two lasers they use to make a pit smaller than you would expect. There's not a whole lot of information in the article to understand the details. Also, I'm just a guy who works on lasers, getting C's in my optics class currently so take it with a massive dose of salt
yeah that's why I never get excited about this news. That's awesome they're working on it, but it's in a lab, in a controlled environment, with each one costing more than I make in a year probably.
When I get excited is when they have manufacturing down and they want to start making them en masse. Something like this in a lab? Eh a decade away. They're talking about it at their marking reviews and it's in manufacturing? Now I'm excited
I have some questions:
What read/write speeds?
Is it even writeable in a reasonable way? (CD/DVD burners blast your disk with a laser and "cook" darker patches that are your data)
How long does it live on this disk?
I had a bad habit whenever I would go to reinstall an OS to just copy the entire user folder into one of many places on my largest hard drive. I had at least 4 or 5 of these. So tons of it was cache files.
Storage is dirt cheap. Just add more. IT at work bugs out their eyes when I talk about adding more storage space. I have more at home than they do in the office. Lol.
I’ve been buying used 8TB HGST drives from eBay. Dirt cheap and haven’t had one fail yet.
I’ve always been one for moderation. Plus, the big issue was just being able to find all of it. Everything was so scattered and in some cases I had half my Steam library downloaded onto two or three of the hard drives, all outdated.
Now I know where it all is and can easily back it up, where before I only had one copy of a lot of my most important files.
This is actually what lead me to set up a software RAID - my family is primarily Windows and I didn't want to remember if the files were on D:, F:, G:, K:, etc. I'd rather have a root folder that's extendable.
I’m a big advocate of unraid servers. Mix
And match any size of drives you have available into a single large NAS with protection against drive failures. You can use old pc hardware you might have lying around. It’s commercial software but you can demo it for free. It’s good enough that I own two full pro licenses.
Any good tools to deduplicate files? Got a bunch of images with different names, on different folders, sometimes with different resolutions, spread out across 10 external hard drives that I need to go through...
This was done by hand on my end. If I had needed to do it procedurally, I’m not sure what my approach would have been, but the direction I’d probably head in would be to look into finding duplicate MD5 hashes.
Don't have any pictures or videotapes from childhood, teen years or even now. Right now, I have some free music downloaded from Bandcamp from some of my favorite indie artists, and that's what I treasure for now. And a few torrented copies of novel I've yet to start reading. And a GBA state files for Final Fantasy 6 from my use of PC emulators, which is probably corrupt by now, as the SD card lives in my phone
datahoarder
Oldest