With the most interesting major tech announcements of the way for a while, and with a new laptop easing my mind in the way of "how will I write when the Surface 3 dies?" I've had time to think about other things. Not that I have, of course.
Instead, I watched YouTube, and in the back of my mind, I was thinking about the thing that deep down, we all know I really want to do: Save every YouTube video to local storage so I can fall asleep to the sweet, dulcet tones of people recording their progress in Cities: Skylines. And then watch those files again later when I want to see what happened.
Video, both legitimately downloaded into my iTunes library and from things like podcasts, isn't the only thing that eats disk space on my systems. About ten years ago when data was much smaller, my solution to this problem would be to burn a new disc every month or so with data I wanted to keep but didn't need on my disk any more. It worked out well because with the slow Internet connection I had and the relatively slow rate at which I created or otherwise acquired data, there wasn't an awful, one or two DVDs (or if I was feeling spendy, one dual layer DVD) was enough.
Today, writeable DVDs and Blu-ray discs exist, but preparing them is as inconvenient as it has ever been, and these discs, which are costly in their write-once form and even more costly if you try to reuse them for backups, have been massively outpaced by the falling costs and increasing capacities of things like external hard disks and USB flash disks. Optical media is often unreliable, long term. Most of the CDs and DVDs I burned in the early 2000s have degraded to the point that it's questionable whether I'll get the data off them. Any other form of relatively capacious external storage device is very expensive and very enterprise focused. The next best thing, DAT320, was more affordable than LTO, although less robust and also now discontinued.
It strikes me that it would be great to have a modern removable data storage format that's more robust than hard disks, bigger than flash drives and blu-ray discs, and ideally fast.
The problem is of course that there are always compromises. You can't, say, build a storage format that's capacious, fast, and cheap. If you could we'd all have LTO tapes at home. I think that the trade-offs are going to be in capacity and in speed. It won't be as fast as real external hard disks or as big as LTO tapes. In trade for being pretty cheap and being treated like external media, I'm imagining it'll either be a new form of optical or magneto-optical media, or some kind of flexible magnetic storage, in the style of something like zip drives, or perhaps Bernoulli. Honestly, I wouldn't even mind if it was massively cost-reduced DAT/DDS media that had at least doubled or quadrupled capacity. (Ideally, the native storage capacity would be 500 gigs or so.)
I think that for most people don't need an awful lot of that. In fact, most home computer backups aren't any bigger than single disks you can buy today, which is 8TB for external disks and 10TB for internals, plus the size of things like Drobos and home focused NAS devices.
But, the thing I want to do is create multiple media sets for my backup of a big server system that I want to then take away for safe keeping. The other thing I want to do is store data in a semi-archival state. External hard disks are big enough, it's easy to duplicate them, but it's also easy to kill them and a certain amount of inactivity will cause them to die.
The other problem with external hard disks for archival is that they cost a lot up front and they're often much larger than the amount of data I want to "archive" at any given moment, and I don't necessarily want to pull my disks out to add data and then later duplicate it.
I think the common thing to do these days if to put data that's being hoarded in fake unlimited cloud storage locations. I suspect that if it were easier to use something better suited to the task, people wouldn't abuse those tools. The key to this is making the device fast enough. It has to be faster than using an average Internet connection to upload files, but it can be slower than a proper hard disk, I think, and making it inexpensive enough to be able to load up on cartridge. It would also be better if as part of an "archiving" solution there was a way to catalog the contents of the media, although if they're not tapes you should also be able to just browse the devices.
I think that at most the mechanism should cost a few hundred dollars, no more than 500 if possible, and the media should be pretty reasonably priced. If in trading off the convenience of flash disks for lower cost of these cartridges, you can get the media down to around $20 a pop, it would start to make a lot of sense for low end archiving and backup applications.
In an ideal configuration, the mechanisms cost a little less and the media is perhaps a little slower, but it holds a lot more in trade for the speed. I think that the "data hoarding" crowd would be fine with something cheap that worked slowly but perhaps worked in a configuration where multiple drives could run in tandem or where there was a cheap loader or stacker would of course be beneficial, but things like that add complexity and cost.
The trouble is that there are a lot of ifs here. At the top end, for people who are building large multi-terabyte disk arrays to alleviate storage problems, you can almost certainly just get a tape drive and eat the cost. At the low end, RDX costs a lot relative to hard disks, but it's a good durable backup option for systems with less than 4TB of storage, and it's a removable system that works with spanning archive. At the far small end, cloud storage systems with a quota of a terabyte or so are often better as a primary or only storage solution, but mistrust in cloud technology often ends up meaning that some people end up with their data stored locally (not a bad thing) with no or insufficient backups.
This technology is not really marketable or something that is likely possible to exist. It conveniently combines the best aspects of LTO and RDX but cheaper than either of them. I think that there is "a market" for this kind of thing but I don't really believe that it's terribly big. In truth, I'm sure it's quite small.
Part of the problem is that there are people who the data hoarder thing. The people who do data hoarding as a hobby often either have the wherewithal to run regular tapes, or are totally opposed to the idea and might not be interested in such a device.