10 years ago I worked at a university that had a couple people doing research on LHC data. I forget the specifics but there is a global tiered system for replication of data coming from the LHC so that researchers all around the world can access it.
I probably don’t have it right, but as I recall, raw data is replicated from the LHC to two or three other locations (tier 1). The raw data contains a lot of uninteresting data (think a DVR/VCR recording a blank TV image) so those tier 1 locations analyze the data and removes all that unneeded data. This version of the data is then replicated to a dozen or so tier 2 locations. Lots of researchers have access to HPC clusters at those tier 2 locations in order to analyze that data. I believe tier 2 could even request chunks of data from tier 1 that wasn’t originally replicated in the event a researcher had a hunch there might actually be something interesting in the “blank” data that had originally been scrubbed.
The university where I worked had its own HPC cluster that was considered tier 3. It could replicate chunks of data from tier 2 on demand in order to analyze it locally. The way it was mostly used was our researchers would use tier 2 to do some high level analysis, and when they found something interesting they would use the tier 3 cluster to do more detailed analysis. This way they could throw a significant amount of our universities HPC resources at targeted data rather than competing with hundreds of other researchers all trying to do the same thing on the tier 2 clusters.
They still happily exist on YouTube- for now. So no point in re-hosting, they'll get squirreled away into the Giant Hard Drive of Doom.
If something happens to the actual archive project in the near future, I'll likely section them up into 20gb pieces and post them out on a torrent someplace.
Lemmy has a few bugs and it's slow. The subscribe button sometimes take a couple minutes to work, and I can't even create a post in the lemmy.world community. It will take a few more months to perfect it.
It's not perfect. But I've noticed a disdain for Lemmy in people for no reason at all. I think it's due to the perceived complexity. If you go to join-lemmy.org, it's a little confusing. Many simply give up trying to decide which instance to join. That's a reason why sites like squabbles are getting so much attention.
Its not a backup if it doesn’t follow the 321 Rules of Backing Up
The 3-2-1 backup rule is a strategy that recommends having three copies of your data backed up. The first copy is your primary critical data backup. The others are two redundant backup copies. You should use two different methods to back up your data, such as local and online backups. Then, you should have one copy designated for disaster recovery.
If you are planning on using a RAID1/5/6 (or SnapRAID), think about used drives.
I've just bought 6 6TB drives for €8/TB (this).
It all depends on what you need to store on those drives.
I thought about it but man that uncertainty still spooks me a bit. Plus I'm looking for larger drives at this point since my current home server is pretty full up on SATA connections and hdd drive space. Maybe one of these days when I can afford a full on rack system.
Never ever going to buy Seagate again after the crap they've pulled on their Exos drives.
They simply decided to completely trash SMART and spin down commands. The drives simply won't give you useful SMART data nor they won't ever actually spin down, you can't force it, the drive will report is as if it was spun down but in reality its still spinning.
datahoarder
Top