datahoarder

drdabbles , in Whatever happened to DNA-based storage research?
@drdabbles@lemmy.world avatar

It wasn't sensible, given the short life of DNA. One of those sci-fi ideas that caught media and technophile attention, but wasn't ever going to go anywhere.

Project Silica appears to be attempting very high density, very long life storage, though.

hazeebabee , in Whatever happened to DNA-based storage research?

Im most excited about the potential for crystal based storage. Right now there is work being done to etch silica glass internally, allowing for incredibly long term preservation and durability. It can even be rewritten, though the tech is definetly best for achival purposes and is being pursued primarily by movie companies wanting high quality storage.

Heres a video discussing it more in deapth:
https://m.youtube.com/watch?v=6CzHsibqpIs

DNA also sounds interesting, though it doesnt seem like a good way of preserving data long term. DNA is very fragile, and seems like an odd route to take for long term archiving.

TCB13 ,
@TCB13@lemmy.world avatar

DNA also sounds interesting, though it doesnt seem like a good way of preserving data long term. DNA is very fragile, and seems like an odd route to take for long term archiving.

Yeah the 5D quartz disk is very cool.

Anyways if you think about storage density DNA isn't that "odd". With DNA you can store dozens of copies of the data and parity checks in a very small space so even if some gets corrupted you can still get it. I get that organic stuff has its limits but the density is just mind blowing.

hazeebabee ,

Density is defintly amazing in DNA, its just so fragile. Even our own bodies have a constant degridation of our DNA... I wonder if they could take that concept and make something sturdier by using slightly different molecules to make up the chains.

Maybe shorter chains with stronger cross bonding & a gentle method of reading the chain could also help?

Its definetly an interesting route & itll be cool to see what happens with it over the next 10-15 years.

litchralee , in Whatever happened to DNA-based storage research?

I recall watching a documentary (on Curiosity Stream maybe? I'm no longer subscribed) on data storage longevity. It covered DNA storage, which I think this PBS video w/ transcript provides more recent coverage of its developments. As well as holographic storage, which I could only find the Wikipedia page for.

As for which one I think might be the future, it's tough to say. Tape is pretty good and cheap but slow for offline storage. Archival media will probably end up all being offline storage, although I could see a case for holographic/optical storage being near line. Future online storage will probably remain a tough pickle: cheap, plentiful, fast; select at most two, maybe.

Shadow , in How to datahoard Lemmy
@Shadow@lemmy.ca avatar

Run your own instance, subscribe to every community, then backup the db.

Deckweiss OP ,

As far as I have searched, I didn't find a way to find "every" instance. Most of the popular ones - yes, but far from all.

Also I did not know that each instance copies all pf the data of all other instances into their own db.

But thanks, I will try it out next year.

andrew ,
@andrew@lemmy.stuart.fun avatar

There's no authoritative list of instances since federation isn't required, but tools like lemmyverse.net will give you a solid list of the ones discoverable from the most well known federations.

mindlessLump , in What's an elegant way of automatically backing up the contents of a large drive to multiple smaller drives that add up to the capacity of the large drive? (on Linux)

You could create a Python script to do this. There is a library called psutil that would help. Basically,

  • iterate over mounted drives and see how much each has available
  • based on these values, iterate over your backup files and separate them into chunks that will fit on each drive
  • copy chunks to respective drives

Would be a fun little project even for a beginner I think.

watson387 , in What to do with extra HDDs
@watson387@sopuli.xyz avatar

WTF is an extra HDD?

isles , in The road has been paved

Excited for you! I'm going from 1x 12tb USB drive to 4x internal 18tb drives. I'm building the NAS from scratch and keeping my other server for its current services (mostly Plex). My parts have been defective though, so it's all just sitting waiting for a replacement mobo.

Nogami , in What guides, wikis, or megathreads are available for those new to archiving and storing data?

I went down the unRAID rabbit hole.

I never came back up, I’m happy here.

TrickDacy , in The road has been paved

What's a data scrubber?

PixeIOrange ,

It scrubs your data

beerclue OP ,

It's a data maintenance feature that amends data in storage pools that are incorrect or incomplete. It works on BTRFS volumes or RAID 5/6 storage pools. It's scheduled to run monthly on my NAS. I guess it started now as I upgraded my drives from 4x4TB to 4x18TB.

ShortN0te , in Remember Seagate’s Dual Actuator HDDs? They’re Back, in SATA Form

The amazing thing about those are that they are halfing the rebuild time. With large drives you get rebuild time of over 24 hours which is actually frightening.

Setup is a one time thing and yes you need to be carefull about it but i bet software support will come as soon as those get more mainstream.

TCB13 , in Remember Seagate’s Dual Actuator HDDs? They’re Back, in SATA Form
@TCB13@lemmy.world avatar

Never ever going to buy Seagate again after the crap they've pulled on their Exos drives.

They simply decided to completely trash SMART and spin down commands. The drives simply won't give you useful SMART data nor they won't ever actually spin down, you can't force it, the drive will report is as if it was spun down but in reality its still spinning.

netburnr , in Remember Seagate’s Dual Actuator HDDs? They’re Back, in SATA Form
@netburnr@lemmy.world avatar

It's a Seagate, i would never buy it. Host (now wd) for life.

empireOfLove ,

Timd to update your criteria, friend. Seagate hasn't been top of the failure stack for like 8 years now. The 3TB scandal era is long since passed. Now it's WD who has been shitting on quality control, sending out faulty SSD's that wipe user data, bait-and-switching HDD customers with a cheaper, much worse performing technology (SMR) WITHOUT TELLING THEM, them basically blowing corporate raspberries at everyone when people complain.

While i agree they were the best, HGST also hasn't even existed as a non-WD product for years....

Sharpiemarker ,

Yep. Seagate have earned their reputation. Pass.

pastermil ,

Care to elaborate?

Sharpiemarker ,

They've had some of the highest failure rates among drive manufacturers.

roawre ,

Care to elaborate? Seagate is one of my favorite brand. And i read a lots of reviews and tech articles before purchasing any components. I am curious to learn about what i have missed about them. Thx

Nollij ,

Not OP, but this comes up regularly.

A lot of people have very strong opinions of brands based on a woefully inadequate sample size. Typically this comes from a higher than expected failure rate, possibly even much higher than expected. It could've been a bad model, a bad batch at manufacturing, improper handling from the retailer, or even an improper running environment. But even the greediest data hoarders only have a few dozen drives, often in just a couple of environments and use-cases.

Very few of these results are actually meaningful trends. For every person that swears by WD and will never touch a Seagate, there's someone else that swears by Seagate and will never touch another WD. HGST and Toshiba seem to have a very slight edge on reliability, but it's very small. And there are still people that refuse to touch them because of the "Death Star" drives many years ago.

It's also very difficult to predict which models will have high failure rates. By the time it becomes clear one is a lemon, they're already EoL.

I avoid buying WD new because of their (IMHO completely illegal) stance on warranty, but I'm comfortable buying their stuff used.

Don't worry too much about brand. Instead go for specs and needs. Follow a good backup strategy and you'll be fine

LeafOnTheWind ,

HGST is a part of WD and has been for quite a while.

But a big part of why the average consumer drive kind of sucks is that there is way more money in enterprise level drives so very little resources get put toward client drives.

Nollij ,

Owned by, yes. Have their operations actually been integrated though? I haven't checked in a long time, but it was still a separate division last time I did.

LeafOnTheWind ,

It's integrated. Only a few things internally are still labeled HGST.

yote_zip , in Remember Seagate’s Dual Actuator HDDs? They’re Back, in SATA Form
@yote_zip@pawb.social avatar

ZFS and BTRFS could update their codebase to account for these (if they haven't already), but I agree that their extra mechanical parts worry me. I really don't care about speed - if you run enough HDDs in your RAID then you get enough speed by proxy. If you need better speeds then you should start looking into RAM/SSD-caching etc. I'd rather have better reliability than speed, because I hate spinning rust's short lifespan as-is.

maxprime , in Lost Doctor Who episodes found – but owner is reluctant to hand them to BBC

If they want it so much why don’t they pay him? Sounds like if it weren’t for him (and the others he seems to allude to) we wouldn’t have this opportunity.

Donjuanme ,

Would you pay someone who stole something from you?

I would've.

maxprime ,

I would pay someone who rummaged through my trash and decades later I realized I shouldn’t have thrown it away.

Glowstick , in Lost Doctor Who episodes found – but owner is reluctant to hand them to BBC

I WANNA SEE IT I WANNA SEE IT I WANNA SEE IT!!!!!

  • All
  • Subscribed
  • Moderated
  • Favorites
  • random
  • datahoarder@lemmy.ml
  • test
  • worldmews
  • mews
  • All magazines