One other thing that has been missed is that because dedupe is (as a rule) carried out as a post-process you need to keep some 'spare' capacity anyway to place yet-to-be-deduped information prior to the dedupe process kicking off, so when you measure your high water mark make sure it's before your daily dedupe kicks in rather than when you wander in with your coffee and the process is long completed. Yes it's a relatively simple point but given that from the comments half the people here didn't even get the basic idea of the article there is some worth in publishing this stuff. As would be expected, if all you're doing is decrementing a reference then you're not oging to save a lot of space by doing so, and so 'traditional' methods of keeping your storage utilisation down aren't going to work with deduped storage. It has to do with reduced referencing, with the reference going from n to n-1 for a given piece of data where n >1. This has nothing to do with garbage collection, with the reference count going from 1 to 0 for a given piece of data. No icon - there is no appropriate icon for this level of moody and bitter. (I'd use the word 'smarter', but in the context of high dollar IT kit sold on golf courses, I'll settle for the much more practical 'less stupid' term.) Hopefully, you have chosen a less stupid vendor for the new system. The only option for deletion then is to migrate your known good data to a new system, and drop your old data store in the shredder.
It's like a gas station burrito that just keeps on giving. Bonus points to the vendor for sheer asshattery if they combine that with a unique file format that means you don't gain any of said de-duplication benefits. There's no use in reference counting if you never decrement the reference count or delete the reference when it hits zero from where it counts. I stopped believing advertisements a long time ago.Īfter all, the content id handed to a third party application, if 'deleted' in the third party app, remains in the filestore index.
I know of several vendors that are quite unclear on the concept, having had their professional services architects and technicians set up systems with no viable plan whatsoever for clearing the space other than the claims printed on the glossy advertisement.
In any case, to be quite honest, we in the real world aren't much interested in how easy your lives are.Īnd sometimes it isn't even the managers' fault
There are a lot of naive decision makers about who will believe anything the vendors (but not their own technical staff) tell them, and not make the connection that introducing the technology means that the old "disk space panic - lets delete something " bit won't always work and they will need to build in rather more effort in terms of proactive monitoring and management, and keep a bit more contingency ad not put off buying the new storage until the end of the FY.Īnd for those serial commentards (flamentards?) who are already ready hitting the comment button or the down vote with "they should be doing that anyway", why don't you include in your posts the wonderful organisations you work for where there are no short sighted managers or penny pinching bean counters so the rest of us struggling with such know what companies to apply for jobs to. For those who are being fed deduplication Kool Aid.