My father told me he wanted to make USB flash drives of all the scanned and digitized family photos and other assorted letters and mementos. He planned to distribute them to all family members hoping that at least one set would survive. When I explained that they ought to be recipes to new media every N number of years or risk deteriorating or becoming unreadable (like a floppy disk when you have no floppy drive), he was genuinely shocked. He lost interest in the project that he’d thought was so bullet proof.

  • mox
    link
    English
    88
    edit-2
    24 days ago

    I explained that they ought to be recipes to new media every N number of years or risk deteriorating or becoming unreadable

    This is important, and for some media, it should be more often than that.

    People forget that flash memory uses electrical charge to store data. It’s not durable. If left unpowered for too long, that data will get corrupted. A failure might not even be visible without examining every bit of every file.

    Keep backups. Include recovery data (e.g. PAR2). Store them on multiple media. Keep them well-maintained (e.g. give flash drives power). Mind their environment. Copy them to new storage devices before the old ones become obsolete.

    It’s funny that with all our technology, paper is still the most durable storage medium (under normal conditions) that doesn’t cost an arm and a leg.

    • @SkyNTP@lemmy.ml
      link
      fedilink
      English
      2424 days ago

      It’s funny that with all our technology, paper is still the most durable storage medium (under normal conditions) that doesn’t cost an arm and a leg.

      Sophistication often creates fragility. The human mind marvels at sophistication naturally; appreciation for resilience usually only comes after that fragile thing has broken. Of course it’s too late by then.

      All them young whipper snappers will continue to learn these life lessons the hard way, it seems.

    • @BehindTheBarrier@programming.dev
      link
      fedilink
      English
      1524 days ago

      If I had a cent every time an artist on patron had their computer die on them and lost works in progress or all their old stuff… I’d afford a few coffees.

      • @slacktoid@lemmy.ml
        link
        fedilink
        English
        123 days ago

        Maybe you should have asked for a dollar everytime that happened. Youd be rich I tell ya!

    • @MonkderVierte@lemmy.ml
      link
      fedilink
      English
      5
      edit-2
      24 days ago

      People forget that flash memory uses electrical charge to store data. It’s not durable. If left unpowered for too long, that data will get corrupted.

      Yeah, but the link in the article, strict checks and no data loss over 52 weeks. Not neccessarily in USB sticks though. And sure, backups.

      • mox
        link
        English
        8
        edit-2
        23 days ago

        That number is a single manufacturer’s performance target. It is not a guarantee of results. You might be able to get Intel to replace an SSD if it corrupts data in under 52 weeks (assuming you notice it) but your data will still be gone.

        Hardware performance can and does vary by manufacturer, model, and production run. Even the nominally identical cores within a single CPU have slightly different operating limits. YMMV.

        Note also: the 52 week target you quoted is halved for every 5° rise in temperature.

  • @mctoasterson@reddthat.com
    link
    fedilink
    English
    7623 days ago

    This is where “piracy” is actually the industry’s saving grace. Decades or centuries later, will record labels exist and be well-managed (and flush with cash) enough to preserve archival copies of their artists catalogs? Probably not.

    Will obscure weirdos exist all around the world on Usenet, IRC, or seeding torrents? Possibly.

    • @frostwhitewolf@lemmy.world
      link
      fedilink
      English
      2723 days ago

      What is really being discussed here is archiving of master recordings and session files. The publically avaliable releases themselves aren’t really in jeopardy. Orthough piracy probably does provide an extra layer of security to more obscure releases.

      • @tunetardis@lemmy.ca
        link
        fedilink
        English
        1623 days ago

        I thought I read somewhere that when they were making one of the Toy Story movies, there was some catastrophic data loss that nearly tanked the whole production. But then one of the animators came back from maternity and said wait, I think I have most of it synced to my home server? And the next thing you know, John Lasseter himself is barrelling down the highway to her place and it turned out yeah, she did have it.

    • @curry@programming.dev
      link
      fedilink
      English
      2423 days ago

      Many films and tv programs survive only thanks to a total stranger keeping their own copy. For a long term survival of any media it has to be copied and distributed far and wide.

  • Shimitar
    link
    fedilink
    English
    71
    edit-2
    24 days ago

    There is no “write and forget” solution. There never has been.

    Do you think we have ORIGINALS or Greek or roman written texts? No, we have only those that have been copied over and over in the course of the centuries. Historians knows too well. And 90% of anything ever written by humans in all history has been lost, all that was written on more durable media than ours.

    The future will hold only those memories of us that our descendants will take the time to copy over and over. Nothing that we will do today to preserve our media will last 1000 years in any case.

    (Will we as a specie survive 1000 more years?)

    Still, it our duty to preserve for the future as much as we can. If today’s historians are any guide, the most important bits will be those less valuable today: the ones nobody will care to actually preserve.

    Citing Alessandro Barbero, a top notch Italian current historian, he would kill no know what a common passant had for breakfast in the tenth century. We know nothing about that, while we know a tiny little more about kings.

    • @futatorius@lemm.ee
      link
      fedilink
      English
      1523 days ago

      Do you think we have ORIGINALS or Greek or roman written texts?

      In some cases we do. For example, the documents at Pompeii, those at Qumran, and other rare instances where documents have been preserved from the time they’re written. It’s also true that we have far more copies than originals, and that we’ve lost most of the works of many ancient authors.

    • @endofline@lemmy.ca
      link
      fedilink
      English
      723 days ago

      There is: mdiscs. Allegedly 1000 years durability even in Blu-ray format. Should be good enough for most important things. The best tapes AFAIK 30- 100 years

      • @curry@programming.dev
        link
        fedilink
        English
        623 days ago

        Problem is how to read the disk, especially after generations. Will they retain the knowledge to build and operate a device for this?

        • @endofline@lemmy.ca
          link
          fedilink
          English
          623 days ago

          That’s always problem for any type of media. Including the tape which keep changing generations and only few recent are supported for reading. I still have blue ray reader / writer though

          • @thawed_caveman@lemmy.world
            link
            fedilink
            English
            1223 days ago

            It’s even the case for physical media, like paper and carved stone, because over a long enough time people forget the language that they were written in. Historians had to teach themselves how to read ancient egyptian, and off the top i think a lot of Maya inscriptions are still a mystery.

        • @LarmyOfLone@lemm.ee
          link
          fedilink
          English
          323 days ago

          I wish there was a cheap and millennia-long lasting microfilm you could transfer books to. A projector is a pretty simple device to operate. Hmm that reminds me of “Last Words (2020)”.

          • @Hamartia@lemmy.world
            link
            fedilink
            English
            423 days ago

            Microfilms used to be sold as having a life expectancy of up 500 years. But in my experience they were a pain to use and the machines costly to maintain. The films would tear regularly too. Also the quality of the recorded image could be very poor sometimes.

    • @nyan@lemmy.cafe
      link
      fedilink
      English
      623 days ago

      Do you think we have ORIGINALS or Greek or roman written texts?

      We do have originals of some much older texts, though (cuneiform on clay that was fired after impression seems to be a pretty good archival medium, overall). We’d probably have a lot more original Greek and Roman documents if they hadn’t been destroyed in wars and other disasters, or recycled for various purposes. There’s a big survival rate difference between documents that receive basic care throughout their lives—no rough handling handling, minimal direct sunlight exposure, and some degree of temperature and humidity control in the storage area—and those left to fend for themselves. That’s why old documents in surprisingly good condition sometimes turn up in caves, which tend to have constant temperature and humidity levels.

      (But, yeah, current electronic media doesn’t have much chance, with select optical disk media stored under carefully chosen conditions offering the best chance for your files being retrievable decades later, if you can find a drive to read them on.)

      • @merc@sh.itjust.works
        link
        fedilink
        English
        223 days ago

        Yeah, I was going to say that we know that Ea-nasir’s copper was shitty.

        Obviously not everything from 1750 BC survived, but we do know that certain mediums are more likely to stand the test of time than others. Something physical with the writing carved in? That will probably last. Something with pigment on vellum, that won’t be quite as good, but you can store a lot more information per kg. Something involving bits? That won’t last even a quarter century. Something involving bits stored using magnetism and retrieved using mechanical motion? Good luck keeping that for even a decade.

        But, the thing we’ve shown will 100% stand the test of time is keeping the information flowing, though at the cost of some degradation. In the past, this was one generation telling stories to the next. When that happens, not only does the information get passed on, the language used is subtly updated in time with the evolution of the language. You don’t need to learn Akkadian cuneiform to read it, it’s available in whatever the modern language is. Similarly, if digital files keep getting passed around, it doesn’t matter if the original came on a floppy disk, and floppy disk readers are now gone. The file exists, stored in whatever medium is current. But, you get degradation with this process too. Music might be turned into mp3s with some data getting lost. Photos might be resized, cropped, recompressed, etc.

        If I wanted something to be preserved exactly as-is for centuries, I’d carve it into a non-precious metal (so nobody melted it down). If I wanted something to be easily accessible for centuries, I’d try to share it as widely as possible to keep it “in motion” and in a format that was constantly up to date.

    • Flying Squid
      link
      fedilink
      English
      323 days ago

      But we do have originals of many other texts. Like vast, vast amounts of cuneiform texts because they were usually pressed into a clay tablet and then baked.

      All but four Mayan paper codices were burned, but the Mayans loved carving their stories into rocks and making those rocks part of their cities’ architecture, so we still have a lot of their textual information. Same with the Egyptians- they wrote a lot on papyrus, and most of that is lost, but they also carved and painted all over tombs.

      The secret is not to keep copying the texts. That introduces errors and those errors can compound. The secret is to preserve the texts in a medium which will only degrade over exceptionally long periods of time (compared to a human life, anyway).

      If you had a device which could carve stored data into stone and make it retrievable again, you could potentially preserve that data for thousands of years.

    • Unbecredible
      link
      fedilink
      English
      324 days ago

      Sad little human. I have written my treatises into the warp and weft of reality itself. I have twisted my curiosity into the folds of your DNA and stamped my waxing madness into the ragged edges of the telomeres that mark your days as numbered. I have made of the stars a celestial QR code that burns across the skies of every planet, that burns across the eyes of every ape who stares into the night and asks “why?”. I announced The Work with a bang of gas and light and awe and set time itself into motion so my scripture could expand eternally into the infinite, benighted expanse.

  • Toes♀
    link
    fedilink
    English
    5424 days ago

    Yeah if you’re looking for long term it needs to be archival media. Many people think the flash drive will hold it forever but they are potentially the most fickle.

    • @leisesprecher@feddit.org
      link
      fedilink
      English
      2024 days ago

      But what actually is “archival”?

      Like, what technology normal person has access to counts at least as enthusiast level archival?

      Magnetic tape, optical media, flash, HDD all rot away, potentially within frighteningly short timeframes and often with subtle bitrot.

      • The Snark Urge
        link
        fedilink
        English
        1624 days ago

        M-DISC, at a guess. The media would last long enough at least for grandkids, who will have bigger things to worry about.

        • @thejml@lemm.ee
          link
          fedilink
          English
          1024 days ago

          Don’t forget, you also need drives that work that long and connect to computers or some other device to utilize the bits, and the bus they use must be available and working, and the disk format they’re written in must be readable, and the images themselves encoded with an algorithm that we still have access to, etc. it’s not just the media.

          I think it’s possible, thanks to the retro enthusiasts, we still have access to some things from the 70s and 80s, but they’re getting fewer and fewer, especially in a working state. That’s only 50yrs ago. What happens when you want to go 100? Or 500? A few thousand? We are familiar with journals from the Civil War, and have found items and notes from Egypt, Roman, and Ancient Greek civilizations, how can we preserve what happened in the currently information rich time we live in, for future generations? Especially as much of it migrates online to blog posts and social networks and news sites that eventually shut down due to corporate issues or shifting internet traffic?

          • chiisana
            link
            fedilink
            English
            024 days ago

            Upload it to the cloud and make it someone else’s problem to deal with keeping up with the physical medium changes. Then your descendants only have to worry about figuring out how to deal with an outdated file format they can no longer open… and even when they can finally open it, it’d be super low quality… just like how we have to squint really hard at videos from VCDs now days.

            • @thejml@lemm.ee
              link
              fedilink
              English
              824 days ago

              There have been plenty of cloud services that have shut down and taken their data offline. And plenty of current ones deleted data after users have gone inactive. Or require constant payments to keep accounts active. Cloud, as it exits now, is not the answer to the archival question.

              • chiisana
                link
                fedilink
                English
                022 days ago

                You’ll be very hard pressed to find anything else that’d out last the day when all three of AWS, Azure and GCP shutdown and take their data offline.

                I get it though, Lemmy doesn’t want to admit these services exist other than to dunk on them in the most anti-corporate fashion… so continue to pretend such is the case!

                • @thejml@lemm.ee
                  link
                  fedilink
                  English
                  122 days ago

                  They take your data down pretty quick when you die and stop paying for it. And as much as we all want to think AWS and GCP and Azure are sticking around forever there’s no reason at this time to believe they will be around in 100+ years.

        • Angry_Autist (he/him)
          link
          fedilink
          English
          124 days ago

          Holographic storage is a fluff project, the resolution we’d need to match modern density is simply to narrow to be done optically. I mean it sounds fun but will never be practical

          • @whocares314@lemmy.world
            link
            fedilink
            English
            023 days ago

            That’s just straight up factually incorrect. From the link:

            As a storage technology, Silica offers volumetric data densities higher than current magnetic tapes (raw capacity upwards of 7TB in a square glass platter the size of a DVD), and using beam steering of the laser beam, we’re able to achieve system-level aggregate write throughputs comparable to current archival systems.

      • @tibi@lemmy.world
        link
        fedilink
        English
        4
        edit-2
        24 days ago

        Hard drives offer the best price/capacity ratio, but they need to be powered periodically (at least once or twice per year). As with any other storage medium, include parity data and have multiple backups to avoid data loss.

        Tape is too expensive.

        Optical media can also be pretty good as long as you get discs made from inorganic materials and store them properly. M-disc is supposed to last like 100 years. The biggest problem is that they are on the path to obsolescence and optical drives may stop being manufactured. Also, it’s a good idea to check on the condition of the discs periodically and redo any that shows signs of degradation (probably a good idea to replace non-M discs every 10 years regardless).

        But regardless of the media, there is no archival method that doesn’t require active maintenance, like periodically checking the data, ensuring you have multiple backups, refreshing any aged media.

      • @MangoPenguin@lemmy.blahaj.zone
        link
        fedilink
        English
        -324 days ago

        Like, what technology normal person has access to counts at least as enthusiast level archival?

        Cloud storage? Store it on 2 different providers like B2 and iDrive or something, pretty low complexity.

        • @leisesprecher@feddit.org
          link
          fedilink
          English
          1524 days ago

          Is it? It’s rather expensive and would you really know, if the data is gone or corrupted?

          You’d have to download every single file in certain intervals and check it. That’s not really low complexity.

          • Dave.
            link
            fedilink
            English
            124 days ago

            I’ve got photos in Flickr dating from 1999 onwards. Ten thousand or so of them, and a couple of the early ones are now corrupted.

            But they are my “other backup” for Google photos so I don’t mind too much. I also have a USB Blu-ray drive at home that I use to periodically burn M-Discs that I hand out to a few relatives.

            That’s about as good as I can conveniently do for backup, and it’s probably better than the single-point-of-failure box of negatives that my parents have in their cupboard.

            • @leisesprecher@feddit.org
              link
              fedilink
              English
              223 days ago

              And who does that?

              I think you don’t really get my point. I’m not arguing that there are no ways to archive data. I’m arguing that there are no technologies available for average Joe.

              It is hardly a good strategy to basically set up half a datacenter at home.

              • @MangoPenguin@lemmy.blahaj.zone
                link
                fedilink
                English
                123 days ago

                Yeah that’s fair.

                Common cloud storage such as google drive should be pretty resilient for the average person, data stored there is replicated in multiple data centers and verified with checksums, and it provides a trashcan and versioning in case of accidental deletion.

        • Zoot
          link
          fedilink
          English
          1024 days ago

          You have to hope neither goes out of business either.

    • @MrSoup@lemmy.zip
      link
      fedilink
      English
      -1224 days ago

      USB-A is best bet today, will live longer than other formats and USB adaptors will still exist when USB-A will disappear entirely.

        • @MrSoup@lemmy.zip
          link
          fedilink
          English
          -8
          edit-2
          24 days ago

          Werent we talking about usb flash drives?
          Since usb flash drives use usb, I think we can keep using them to store data in long term, rather than using floppy, cd or other analogic archive.

          • @floofloof@lemmy.ca
            link
            fedilink
            English
            13
            edit-2
            24 days ago

            The problem is that USB flash drives don’t keep their data intact for very long when they’re powered down. It lasts long enough for everyday use, but not even as long as a hard disk for archival.

            • Dave.
              link
              fedilink
              English
              824 days ago

              when they’re powered down.

              There’s no periodic cell refresh in flash memory like there is in DRAM. When USB sticks are plugged in, all you are doing is powering up the flash chip and interface ICs.

              You’d have to read a block then write it back to actually refresh the stored charges in the cells.

              • @floofloof@lemmy.ca
                link
                fedilink
                English
                224 days ago

                Do SSDs do that automatically in the background, or is all the data I’m not actively refreshing gradually rotting away?

            • @MrSoup@lemmy.zip
              link
              fedilink
              English
              424 days ago

              Oh, gotcha. Was thinking about the peripheral type support rather than its actual lifetime.

  • @TechSquidTV@lemmy.world
    link
    fedilink
    English
    3124 days ago

    Im really hoping, waiting, for a good dense long-term storage medium. It doesn’t have to be fast, but large, cheap, and durable. I want a way to backup my plex library, or even, daily backups of documents and project files, and I don’t want to think about them ever again.

    • @SkunkWorkz@lemmy.world
      link
      fedilink
      English
      1524 days ago

      Tape is cheap and durable if you store it properly. Except the tape drive is expensive af.

      Microsoft is working on glass storage. A glass plate can last 10,000 years according to Microsoft. Hopefully that tech will get miniaturized and available to consumers within our lifetimes.

      • Flying Squid
        link
        fedilink
        English
        823 days ago

        As a former audio engineer in the days where we still used it- tape can rot.

          • Flying Squid
            link
            fedilink
            English
            623 days ago

            It seemed like the person I was talking to didn’t. The implication was that tape was viable as long-term storage. It isn’t. I’ve seen tapes rot after a year. DATs were especially prone to that, but even things like 2" multitrack audio tape can go bad that quickly.

    • @BluesF@lemmy.world
      link
      fedilink
      English
      324 days ago

      Punch cards? Stored correctly there’s no reason they couldn’t last many human lifetimes. But… Yeah it’ll take a while to encode everything.

      I would have thought that with modern technology we could come up with something like punch cards but more space/time efficient. Physical storage of data - only one write cycle of course, but extremely durable. Even just the same system as punch cards but using tiny, tiny holes very close together on a roll of paper. Could be punched or read by a machine at high speed (compared to a regular punch card, presumably still Ber slow compared to flash media).

      • @futatorius@lemm.ee
        link
        fedilink
        English
        523 days ago

        Paper doesn’t last centuries. Anyway, punched cards don’t have a storage density that’s adequate for modern data volumes. You need something that’ll durably store nanometre-sized marks.

    • @LarmyOfLone@lemm.ee
      link
      fedilink
      English
      2
      edit-2
      23 days ago

      M-Disk is rated to last like 100 1000 years. They are also working on a 125 Terabyte CD. Optical storage is the way to go.

        • @LarmyOfLone@lemm.ee
          link
          fedilink
          English
          323 days ago

          Well yeah but BluRay is still much more expensive and smaller capacity. Lets hope this new 125TB disk works out

        • @kalleboo@lemmy.world
          link
          fedilink
          English
          2
          edit-2
          21 days ago

          The main cause of bitrot in older disks is the organic dyes fading (aside from REALLY cheap disks where delamination was a problem), whereas M-Disc uses an inorganic carbon material

      • @feedum_sneedson@lemmy.world
        link
        fedilink
        English
        323 days ago

        Was it sapphire or something? But one and done. I wonder if you could just keep writing and just “cross-out” the old stuff with that kind of capacity.

        • @LarmyOfLone@lemm.ee
          link
          fedilink
          English
          123 days ago

          Not sure what sapphire means, but here is the article. Just appending records and differential backups would seem to be the way to go.

          the new optical disks are claimed to be “highly stable so there are no special storage requirements.” The researchers tout an expected shelf life of 50 to 100 years

    • @SidewaysHighways@lemmy.world
      link
      fedilink
      English
      224 days ago

      It certainly would make planning the dang home lab easier. Im in a small place! I don’t have room for all the stuff i wanna play with!

      • @futatorius@lemm.ee
        link
        fedilink
        English
        323 days ago

        The commonly used optical disk technologies degrade over time. CD-RW more rapidly than CDR. It’s even worse for higher-density media.

        • @XTL@sopuli.xyz
          link
          fedilink
          English
          123 days ago

          And for bigger data sets, the capacity isn’t there. And writable media is getting more rare. Probably because of the same reason.

      • @ZILtoid1991@lemmy.world
        link
        fedilink
        English
        123 days ago

        Depends on the disk technologies.

        CDs are prone to flaking, otherwise most disk are suspectible of oxidation (disk rot) if stored improperly. M-Disk is a long-life variant of both DVDs and Blu-Rays, although more expensive. However, write-once disks are very ransomware-resilient, and I recommend to add a write-once media to any proper backup setup.