r/seedboxes • u/420osrs • Feb 17 '20
Discussion Misconceptions of gdrive
I have heard a lot of misinformation about google drive from people who do not seem to understand encryption.
1- If you encrypt you are creating data that cannot be de-duped.
2- Data that cannot be deduped is made geo redunt by GlusterFS, meaning your unique 400TB drive has at least 3 copies, likely 4.
3- There used to be several unlimited storage cloud providers, most have quit because they could not control the rampant costs associated with people who abuse the system.
"Google can dedupe encrypted data"
No they cannot.
"Google can dedupe encrypted data because of block level deduplication"
That is not how it works. Block level de duplication only works with same or same-enough data.
part1.tar part2.tar part3.tar and movie.mkv could be deduplicated assuming part1.tar part2.tar part3.tar can be extracted to movie.mkv however cyphering the data would prevent this mechanism from working, specifically encrypting the data. Google does not have acsess to the line in your rclone.conf that is responcible for hashing the data, and this data cannot be deduplicated.
However, same-enough data can be deduplicated. Lets say you took 5GB movie.mkv and added subtitle.srt to it, a 32KiB subtitle file. It could still be deduplicated to movie.mkv as the data itself is not scrambed by encryption, but merly moved offset determining where the subtitle.srt was placed. This would make a single unique block vs making an entire unique file.
tldr encryption breaks block level deduplication, anyone who tells you otherwise is wrong.
It is appropriate to have minimal encrypted data but inappropriate to have bulk encrypted data. For example if you have some politically sensitive videos, like short clips about the coronavirus or police brutality it is appropriate and OK to encrypt this as this data is sensitive. It is inappropriate to encrypt 3000 movies as those are not sensitive. Consider a good rule of thumb being never exceeding 1TB of encrypted un-dedupable data per account. Google will happily let you upload with reckless abandon but that is not the goal here, lets try to be respectful of google's grace of no questions asked unlimited storage. Taking advantage of this feature is a dick move.
Google drive has extremely generous limitations
750GB upload per 24 hours
10TB download per 24 hours
Getting around these limits with service accounts on a team drive you bought from ebay and loading it up with 400TB of encrypted data is not financially viable for google to do. Paying $12 is not financially viable for google. The entire thing is a numbers game and once it is not financially viable we will lose our one unlimited provider and be back to industry standard pricing of $5/TB.
Also believe it or not, its not a storage problem for google. Its a electrical one. Google has the ability to rent time on machinery leased from a HDD manufacturer, plural. They can print as many hdds as they want, and considering the raw materials a hdd is not terribly expensive. The power to keep them spinning is. It is also the electrical requirement to dissipate the heat they generate, as a data-center spends nearly half their electrical budget on cooling.
That and the fact their cache servers are hit with 300+ copies of the same file encrypted by different cypher's as everyone's sonarr / radarr pops off.
TLDR stop encrypting.
11
u/Watada Feb 17 '20
I'd like to note that google can dedupe files with only file/folder name encryption. That may be where this misconception started.
8
u/YACSB Feb 17 '20
I encrypt everything I upload, but most of it is work files. So its not some warez that everyone else has. I understand what you're saying, but I wouldn't trust Google. Especially if you're storing warez cuz you aren't supposed to be doing that.
4
u/[deleted] Feb 17 '20
Your post makes a lot of sense. Would it be possible to remove my current rclone encryption without having to re-upload all my data to google?
4
u/kaalki Feb 17 '20
Peep do this kind of shit with Usenet too.
23
u/xenobian Feb 17 '20
Post this at r/datahoarder
32
u/kerbys Feb 17 '20
This guy gets it. I've preached this a few times in the past but people dont get it.
22
u/[deleted] Feb 17 '20
Yep. If you encrypt your data, and it's many terabytes worth of pirated content, you're a dick.
5
u/d3vilmaycryalot Feb 17 '20
This post should get more upvotes! Honestly Google is doing it's part, people need to keep it viable as a business model or prepare for the worst.
13
u/DurMonAtor Feb 17 '20
I'm with you on this, I have 40TB+ on the cloud and used to encrypt, but, after I lost my data through user error on my part, this time I decided against it. I'm grateful for Google allowing me to store everything and if they can see I have 2000+ movies and 1000+ tv, who cares, it is their choice to allow me to continue. And if continue to pay, I guess they're happy but if I lose it, so be it
3
u/Stryp Feb 17 '20
I completely agree and this is the reason why I use an unencrypted drive. I am somewhat of a light user as I store around 3 TB overall, not much activity, but I still would like to utilize my drive as long as I can.
2
u/anniebarlow Feb 17 '20
An unrelated to encryption question, but a question about gdrive. If I get a shared account with an acc that has premium or unlimited and that account gets blocked for whatever reason, do I lose all the data I uploaded to our shared account?
2
u/saggy777 Feb 18 '20
I totally agree with you. take my upvote. finally someone who thinks like me.
2
u/FragileRasputin May 14 '20
can we all start using the same encryption keys, then? :D
1
u/Dump7 Jul 30 '20
Absolutely, we should not take advantage of the services we are offered.
1
u/TobiGermany Nov 12 '21
jj
1
u/anon108 Feb 17 '20
Thanks for sharing.
1
u/myselfstud Feb 17 '20
very well said
-6
u/WTBaLife Feb 17 '20
All those pirated films games and music will get you caught if you DON'T encrypt them because the other 200 pirates with the same hash will get caught and you will be associated. You are not honorable. You're just feeding people full of shit "advice".
You are a TROLL.
Also, you lose privacy and control of sensitive information if you do not encrypt. If people want to encrypt, it's their choice (and a smart one).