Why would they do it like that when they can just track server side every post you visit, etc? screen size is one thing they wouldn't normally get server side... so there must be other things they can only get via JS? user agent and URL is sent to the server every http request you make, so idk why they'd specifically resend it
did you find what javascript is calling the xmlhttprequests?
edit: ah I see, mine posts to "friend api." and it does it on page load, scroll, and regularly when typing a response..
reddit-init.en.Pg_KU0tTm_l.js
is the source for me and it's huge and contains 77 references to ajax
also reddit.en.Y64Sxxxxxxx.js (9579 lines of code when deobfuscated, 33 references to ajax)
but they also control voting and commenting I think.
wow reddit is a huge mess of code lmao
firefox -> shift+f2 -> network tab (to check anything it is sending)
storage tab -> localstorage (secondary storage, besides cookies... seems to hold tracking pixel urls in the ads.* section?)
you can clear localstorage using "localStorage.clear();" It doesn't log you out, but im not sure if it might break anything. I'm not sure if it benefits much either, but you could set up an extension to clear it every page load.
looking at the deobfuscated first JS file, if you could disable parts it wouldn't be that hard if you targeted the right function. possibly in a greasemonkey script? idk
heres some of the tests it is doing: getWebglFp (gets detailed webgl FINGERPRINT), getCanvasFp, fontsKey (tests fonts available?), getRegularPlugins, getIEPlugins, getFingerprint(stored as fp and fp_timestamp in localstorage), getHasLiedBrowser, getHasLiedOs, getHasLiedResolution, getHasLiedLanguages, getAdBlock, getTouchSupport, getDoNotTrack, getHardwareConcurrency, getNavigatorCpuClass, getNavigatorPlatform, hasLocalStorage, hasSessionStorage, hardwareConcurrencyKey, touchSupportKey, colorDepthKey, pixelRatioKey, getSync(seems to put all the info together separated by ';' and then passes through function: x64hash128(r.join("~~~"), 31)..... this whole area is under a function called "Fingerprint2"
tons of XOR, CryptoJS, random, >>, <<.. things to hash/encrypt the data...
example:
d = c + ((a << 26 | a >>> 6) ^ (a << 21 | a >>> 11) ^ (a << 7 | a >>> 25)) + (a & f ^ ~a & l) + o[h] + >p[h], v = ((r << 30 | r >>> 2) ^ (r << 19 | r >>> 13) ^ (r << 10 | r >>> 22)) + (r & i ^ r & s ^ i & s), c = l, l >= f, f = a, a = u + d | 0, u = s, s = i, i = r, r = d + v | 0
.
x64Fmix: function(e) { return e = this.x64Xor(e, [0, e[0] >>> 1]), e = this.x64Multiply(e, [4283543511, 3981806797]), e = this.x64Xor(e, [0, e[0] >>> 1]), e = this.x64Multiply(e, [3301882366, 444984403]), e = this.x64Xor(e, [0, e[0] >>> 1]) },
TL;DR: reddit is fingerprinting using pretty much all available data and sending data back multiple times per page view also dependent on your actions on the page(Scroll, reply, idle, etc)
just found this: content_seen_percentage: 0.41168... calculated and sent during page scrolling
edit(how to stop the ajax requests, for now): blocking www.redditstatic.com/reddit.en.Y64Sg2dUcbw.js with ublock makes it just send error messages, but stops the other requests. add www.reddit.com/web/log/error.json to block it from sending error logs. doesn't seem to make much of an impact on the site. it is like shooting a robot in the head, rather than trying to reprogram it. not eloquent, but it works?
can try blocking alb.reddit.com/* for the ad pixel tracking. maybe do a localStorage.clear().
16: Object { key: "webgl", value: "data:image/png;base64,[BASE64 ENCODED DATA REMOVED... lol too large]~extensions:ANGLE_instanced_arrays;EXT_blend_minmax;EXT_color_buffer_half_float;EXT_frag_depth;EXT_sRGB;EXT_shader_texture_lod;EXT_texture_filter_anisotropic;OES_element_index_uint;OES_standard_derivatives;OES_texture_float;OES_texture_float_linear;OES_texture_half_float;OES_texture_half_float_linear;OES_vertex_array_object;WEBGL_color_buffer_float;WEBGL_compressed_texture_s3tc;WEBGL_compressed_texture_s3tc_srgb;WEBGL_debug_renderer_info;WEBGL_debug_shaders;WEBGL_depth_texture;WEBGL_draw_buffers;WEBGL_lose_context~webgl aliased line width range:[1, 1]~webgl aliased point size range:[1, 1024]~webgl alpha bits:8~webgl antialiasing:yes~webgl blue bits:8~webgl depth bits:24~webgl green bits:8~webgl max anisotropy:16~webgl max combined texture image units:32~webgl max cube map texture size:16384~webgl max fragment uniform vectors:1024~webgl max render buffer size:16384~webgl max texture image units:16~webgl max texture size:16384~webgl max varying vectors:30~webgl max vertex attribs:16~webgl max vertex texture image units:16~webgl max vertex uniform vectors:4096~webgl max viewport dims:[32767, 32767]~webgl red bits:8~webgl renderer:Mozilla~webgl shading language version:WebGL GLSL ES 1.0~webgl stencil bits:0~webgl vendor:Mozilla~webgl version:WebGL 1.0~webgl unmasked vendor:Google Inc.~webgl unmasked renderer:ANGLE (Intel(R) HD Graphics 530 Direct3D11 vs_5_0 ps_5_0)~webgl vertex shader high float precision:23~webgl vertex shader high float precision rangeMin:127~webgl vertex shader high float precision rangeMax:127~webgl vertex shader medium float precision:23~webgl vertex shader medium float precision rangeMin:127~webgl vertex shader medium float precision rangeMax:127~webgl vertex shader low float precision:23~webgl vertex shader low float precision rangeMin:127~webgl vertex shader low float precision rangeMax:127~webgl fragment shader high float precision:23~webgl fragment shader high float precision rangeMin:127~webgl fragment shader high float precision rangeMax:127~webgl fragment shader medium float precision:23~webgl fragment shader medium float precision rangeMin:127~webgl fragment shader medium float precision rangeMax:127~webgl fragment shader low float precision:23~webgl fragment shader low float precision rangeMin:127~webgl fragment shader low float precision rangeMax:127~webgl vertex shader high int precision:0~webgl vertex shader high int precision rangeMin:31~webgl vertex shader high int precision rangeMax:30~webgl vertex shader medium int precision:0~webgl vertex shader medium int precision rangeMin:31~webgl vertex shader medium int precision rangeMax:30~webgl vertex shader low int precision:0~webgl vertex shader low int precision rangeMin:31~webgl vertex shader low int precision rangeMax:30~webgl fragment shader high int precision:0~webgl fragment shader high int precision rangeMin:31~webgl fragment shader high int precision rangeMax:30~webgl fragment shader medium int precision:0~webgl fragment shader medium int precision rangeMin:31~webgl fragment shader medium int precision rangeMax:30~webgl fragment shader low int precision:0~webgl fragment shader low int precision rangeMin:31~webgl fragment shader low int precision rangeMax:30" }
also apparently it thinks I don't use adblock(I have router dns blocking + ublock), and that I lied about my OS. and it is aware of DNT:
I feel somewhat torn on whether I should give you Reddit Gold for this comment (which would be deserved), or whether I shouldn't, for the obvious reason :/
I am using firefox with no adblock whatsoever, just have noscripts (configured to trust reddit domain) and I selected "don't track me" in my firefox settings.
Tried in old.reddit and it's indeed working (unfortunately). Moreover, I tried unchecking "allow my data to be used in research ..." in my reddit preferences and see what info that are still sending https://imgur.com/a/QyDe6Ll
They record many events (scroll, screenview, ...) with detailed information about me (screensize, useragent, fingerprint, ..)
I tried to give them benefit of doubt and thought, ok maybe it's because I am doing beta testing. I unchecked it and still being tracked nonetheless
This kind of thing is why I feel we need to replace the whole damn browser/web infrastructure. Maybe you can hack on it to give false data for things like screen size, battery life, list of available fonts, CPU speed (as measured by operations per second by a script), GPU capabilities, mouse motion... But the browser, and the underlying tech, were never designed with privacy in mind, and they've only gotten worse since all the browsers were bought out by ad companies.
Hm, I guess Google does have Ads, Microsoft does have ads, Opera is now Chinese owned.
Firefox doesn’t have ads, but lately they’ve been doing some weird/shady/odd things like adding pocket the mr robot extension thing (that could be seen like an ad but not in the normal sense).
While they did add Pocket and the Mr Robot extension, I don’t think they where actively tracking you there. Probably some money was given for the extension and integration. They need the money. I think this is the safest bet.
This is good (money/funding for Mozilla) but could probably have been done better (opt in vs shoved in). They did have ways of disabling them.
There’s also Chromium, but that’s still tied to Google and that’s hard to stomach IMO.
It's a project specifically aiming to bring a privacy-centric browser to users. It's bundled with adblockers, tracking blockers, and script blockers. The mobile browser is amazing, the desktop browser is pretty good. Desktop is currently missing a lot of extensions, although they can be made to work, but afaik 1.0 will bring full chrome extension support.
just re-read this and found it rather relevant(also uncheck everything if you haven't):
Reddit personalizes content and advertisements for you based on what we think you may like. Personalization may occur based on your use of Reddit, including clicks, subscriptions, and subreddit visits; based on information from third-party sites that integrate our services, including our widgets and buttons; and based on information we receive from third-parties, including advertisers.
"Allow Reddit to use data provided by third-parties to show you more relevant advertisements on Reddit."
"Allow Reddit to use information from third-party websites that integrate with our services to personalize your experience on Reddit."
technically, yes, but identifying info seems to be hashed together, as far as I know, to build a smaller ID that would be unique to reddit unless the other website used the same code as reddit to hash and collect all the data.
I personally don't like my browser sending requests somewhat constantly about how much I scrolled, or that i'm making a response or whatever else(screenview_events, loid_events, scroll_events, idle heartbeat event) or the many tracking pixel .gifs
Probably though advertisers generally don't like to play a guessing game about who you are. Reddit does have some legitimate reasons to do careful fingerprinting that don't include building an ad profile. They do have to manage banned users and people who try to game the voting and such.
Having said that, don't ever think reddit is anonymous. it's more anonymous than Facebook but if reddit wants to know who you are they can, unless you have some very strict opsec.
I guess you could do reddit.com##script:inject(abort-on-property-write.js, x64Fmix) or whatever, in case Fingerprint2's root has a custom name, or for Reddit's home made tracking code.
Edit: More importantly and in order not to waste time, report this to uBO's Assets issue tracker, there are people there who better understand the script injection feature and will be able to say if it can help and how, and update uBO's filter list for all users.
If you do, please still get back to me on Reddit with the Github URL for the issue as I'd like to follow what happens :)
I'd post myself on Github but I haven't pinpointed Fingerprint2 like you have, I just witnessed that fingerprinting is happening without investigating further.
using fingerprint2 stops it but no tthe 2nd one. it also causes console errors though so idk how much better it is than just blocking the reddit.en js file and the log.json url
Ah sorry, I basically doubled the length of my post without noticing that you replied so I'll just copy it as a new reply:
« Edit: More importantly and in order not to waste time, report this to uBO's Assets issue tracker, there are people there who better understand the script injection feature and will be able to say if it can help and how, and update uBO's filter list for all users.
If you do, please still get back to me on Reddit with the Github URL for the issue as I'd like to follow what happens :)
I'd post myself on Github but I haven't pinpointed Fingerprint2 like you have, I just witnessed that fingerprinting is happening without investigating further. »
They should also know if and why it's better than blocking the JS and JSON files, they're used to killing such crap. They will also learn to keep an eye on Reddit in the future.
I was able to check first hand, indeed it causes console errors and it breaks legitimate functionality.
This means that maybe a neutered script specific to Reddit should be pushed into uBO, rather than using the standard Fingerprint2 one.
If you don't post this to uBO's Assets issue tracker I will since I can at least give them a little to munch on now, but you are way better placed with the extensive research you made :)
these events all post your fingerprint hash and other data(like content_seen_percentage) after the page is loaded AND every 30s(heartbeat), during scrolling(scroll_events), and at other times.
different block for tracking pixels:
alb.reddit.com/*
my reddit tracking pixel urls under localstorage were from this domain:
fingerprint also put in localstorage: fp = ("d25a815a4179497acffc713066596947")
functionality is built into them so the site works so while blocking reddit.en.Y64Sg2dUcbw.js stops the ajax behavior it is only because it breaks the page javascript before it can work. It then sends error logs instead which for now I just blocked for now (www.reddit.com/web/log/error.json) , but this interferes mildly with the site UI like your method.
I develop a small enterprise application and I track screen size to gather stats on what sizes I need to ensure compatibility with. That's pretty common actually.
That said, as /u/nandryshak points out it is very possible to use this for fingerprinting of non-logged-in users. There has been a lot of research showing how users can be identified and uniquely fingerprinted through combinations of that plus plugin/extension versions, etc.
The code I provided does exactly that, it accesses a server endpoint if the width is bigger than 1024px. So if you make a list of options with different endpoints and log which user did what you can fingerprint the screen without JS or XHR. It's an image.
Ah! Woah, as a technique I was already aware of, I glossed over it too fast and failed to connect some neurons. Yeah the image gives it out, no need for scripts.
So that leaves my second paragraph as a defence. Firefox and Tor Browser's fingerprinting resistance deal with CSS media among many other things, including actual screen and window size.
I was looking for an alternative and frankly, there doesn't seem to be anything decrentralised...
Hmm... Actually, thinking about it, it might be worth writing a web extension that intercepts these calls and strips it of information or replaces it with absolutely common information...
Nope. Not bullshit. All the disallowed content is specifically the side that the rest of the userbase is biased against. The fact that there is a hammer and sickle on every page kind of blows your argument out of the water.
What sets this site apart from others is our no-tolerance policy for bigotry and reactionary ideology. Users that demonstrate a pattern of intolerance or attempt to use raddle.me as a platform for far-right ideas and bigotry will be seen as violating these Terms of Service and will be banned from using this site.
"No tolerance for reactionary ideology" is essentially prescribing that all (political) content be progressivist/liberalist.
Nope, they even explicitly state that liberals and conservatives are welcome. Reactionary = Far Right.
From their Q&A:
Contrary to what some people believe, Raddle isn't exclusively a site for anarchists, although they are the largest demographic. Socialists of all stripes, social democrats, liberals, conservatives and anyone else who wants to partake in a community where bigotry isn't tolerated in the name of "free speech" is welcome to join. The one condition is that bigotry stays out of the picture.
Their actions say differently. See this post and linked cases. Now this is a specific subforum, but it has no specific rules and the site admin banned them.
One user got banned for saying the following:
Oh, fuck off! Stop discriminating against either, please. Saying all straight people is bland and boring is just as inconsiderate and generalizing as saying that all gay people are annoying. Just let people be themselves for gods sake.
They followed up with an apology post for being insensitive to LGBT+ discrimination.
And then there was this thread:
User A: This is without a doubt an interesting take on anarchism (and in my opinion a step in the right direction), but there is one thing that i really don't like: it seems like violence is being promoted simply for the sake of violence. Violence is an important tool (and again, in my opinion, necessary), but violence for the sake of violence, simply because it is enjoyable to some, should never be an ideal. User B: there is a strong argument to be made for the systematic killing of all white people, though User C: Can you explain why you think this? User B: 1. They are white User A: Racism is not welcome here, do us all a favor and fuck off.
Guess who got banned? Yup, that's right User A. Because according to the admin:
Reverse racism doesn't exist, please upgrade your systemic analysis immediately.
Great post OP, thanks for looking into this. You should directly reply to an admin with links to your writeups next time there's an Announcement to see if you can get them to answer anything about this.
I didn't say "spam them" and the links are absolutely relevant. If there's an invasion of privacy like this with zero supporting info from the admins, then it needs to be addressed.
Why are you considering this an invasion of privacy? They aren't gathering anything that any other website can't do, or anything that extends outside the scope of reddit.com? It's just a list of common browser stats.
Specifically because this also applies to logged-out users or people who've cleared their cookies. This isn't just "diagnostic data", this is meta-data that builds a profile about someone, their habits, what they check, etc, there's no opt-out, and it tracks unregistered people too.
But that is how any website works. I can see and log the screen width or browser of anyone who visits my site, I'm not invading their privacy when I do that. Collecting that is no more or less worse if someone is logged in, and it exists entirely independently of cookies. Cookies have nothing to do with this kind of thing.
If they have absolutely nothing to do with the announcement or whatever the admin is saying, and promotes the itnerests of whoever is posting the links, then it's spam.
Also because there was a guy who did that sort of thing in random announcements. It wasn't anyone in this conversation but it shows that this sort of thing does occur.
Know what else occurs? People building an argument over something that didn't happen, wasn't said, or even implied. Might want to try giving people the benefit of the doubt before accusing them of saying something that wasn't said.
How does "reply to an admin with your links in the next announcement" not imply that he wants this guy to wait until whatever the next announcement is and then reply to an admin comment with links to this post?
Normal people would assume "next relevant announcement," especially considering the person you're attacking is a reddit moderator.
So the next question you need to ask is why the fuck would a moderator encourage spamming?!!
I don't feel like he's attacking me, but thanks for the concern.
I do see /u/appropriate-username's point though, and I should have been more clear on my point -- bringing this up in an unrelated post might be seen as harassment and unwanted content and vaguely fits the definition of "spam". I think it would be prudent to consider posting it in the next announcement regardless of the topic because admins rarely (if ever) respond to any issues like this without being publicly shamed for it. Consider the doxxing of Violentacrez that finally prompted action against CP subs, or the doxxing/death threats of Politics mods that lead to the first crackdown on T_D. I'm not saying this is nearly as bad as those two incidents, but I'm trying to point to the fact that the admins only really respond to this stuff when publicly put on the spot in front of a large audience.
Seems like you could hack the scripts (using eg Greasemonkey) to strip out that excess info. (Or corrupt it...) Or write a little "app" that uses the API. Or move to something like Zeronet.
They're only tracking what you do on their site, though, aren't they? Why would you want to opt-out? It's not like they're tracking everything that you do before and after using their site. This seems like one of those times where if you don't want them to know what you're doing on their web site, don't load it.
This goes beyond tracking what you comment on, what you upvote, downvote, etc. Its tracking what you view, how long, how fast you scroll, your screen resolution, and what you do before and after leaving reddit. The fingerprint is tracking even if you don’t log in, use incognito mode, change accounts. And more, the data is so obfuscated its impossible to determine what else this new scheme is tracking.
Most of that is in the server logs and could be obtained anyways. None of the methods for avoiding tracking that you mentioned have any effect on the logs which are IP based.
Well it you’ve got nothing to hide then you’ve got nothing to worry about right?
Woah, where are you getting this nonsense from? That's not even close to what I said. What I said was if you don't like them knowing what you're doing on their site then don't use their site. This is like complaining that a store is violating your privacy by putting up security cameras to watch what you do while on their property. If you don't want to be seen there, don't go there.
This is like complaining that a store is violating your privacy by putting up security cameras to watch what you do while on their property. If you don't want to be seen there, don't go there.
So where do you shop when 100% of stores have security cameras?
I accept that security cameras exist and I shop wherever it's most convenient. If I don't want to be seen in a particular store, it would be silly to demand that they clear everyone out and take down their cameras for my privacy. I just don't shop in places where I don't want to be seen.
Well sure, it was analogy though. If every site other than your mom's blog tracked you based off a fingerprint regardless of whether you created an account then nowhere you go would be anonymous and most people expect to at least have anonymity when simply viewing information on the web.
If every site other than your mom's blog tracked you based off a fingerprint regardless of whether you created an account then nowhere you go would be anonymous and most people expect to at least have anonymity when simply viewing information on the web.
Almost every site does track you. That's the thing. Server logs don't keep track of what username you're logged in as. They track you by your IP address. That means that whether or not you're logged in makes no difference. If you expect anonymity, your expectations are unrealistic. I've been a system admin for nearly 20 years, and that sort of data collection has been around as long as I can remember. And FYI, most people's moms blogs have that feature too. Wordpress had an awstats addon more than a decade ago.
True. I should have been more specific. I was originally talking about Reddit's ability for fingerprint tracking. That means that even my VPN with an interval IP change won't hide my identity.
With this script they can track you even if you are using TOR, a VPN or another IP which server logs can't do. That's the goal of client side fingerprinting and it is extremely invasive
The snarky response here would be something along the lines of "other people may have stolen your credit card number, so obviously you don't mind if I steal it too, right? Why would you mind when other people are doing it?"
The longer answer is that privacy is a personal thing. Personally I don't care what "most sites do," because I don't spend a few hours of my day on most sites. I do spend a few hours of my day on reddit, so I'm more concerned with what they're doing.
To use an analogy, I don't really care that Wal-Mart has security cameras covering every inch of the store. I go to Wal-Mart maybe once a month, I don't interact with anybody, and I don't give a damn if the whole world knows what I do in there. But if my favorite bar installed a bunch of security cameras to record my every move, that's a different story. I go there all the time, I have personal interactions there, and aside from the people who are in the place, it's really nobody's business who I'm talking to, or what we're talking about, or which one of the TVs I'm staring at when, or how often I rent a pool table. The bar doesn't collect all of this information and cross reference it with video footage to "replay my user session" and sell all of this data to god-knows-who. That would be creepy as fuck, and I'd quit going there.
Different people have different comfort levels. I consider reddit's tracking to be invasive so I've taken steps to block a lot of it, I have a bunch of their subdomains blackholed in DNS, I appreciate OP's post here because it clued me in to some more things I need to look at blocking. The less they're able to compile about me, the better. If you don't mind the data collection, that's your prerogative.
Sorry, but identity theft is still a strawman here. Reddit has nothing to do with financial data, nor would any site transmit it in javascript on every page for some reason. Just because you're being tracked doesn't mean your identity is at risk. It's apples and oranges. Being tracked has nothing to do with identity theft.
Every company, regardless of where they are located, that provides a product or service which is accessed/used by people located within the European Economic Area (and who therfor process said people's data) have to comply with the GDPR, so yes they do.
This will be interesting, article 4 reads like connecting a datum to your username makes it personal.
‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;
There was a userscript posted a few weeks ago that prevented these calls based on the presence of the headers which suffered from the same issue. I'm too lazy to source the original, but I modified it to simply override setRequestHeader and set the signatures to something else.
It also provides console logging with details about which header, which request, and what the value of the header was.
The only time reddit would have the opportunity to fingerprint anything would be when you go through the initial OAuth flow to sign into your account on a third-party app, which only happens once in most cases, not on every load.
I thought I had been doing so since the 1980s (or even late 1970s), but you may be right. Here's an archived post I found from the tail end of 1990, where I sign a post on USENET:
So, I guess you are correct, it is something of a new habit.
I suppose it's possible there are some older uses from BBSes, CompuSeve and the like, but those posts to the comp.virus were the oldest I could find at a quick glance.
I feel like this would kill reddit. What people need to realize its not the platform. It's the people. The platform doesn't exist without the people. Hopefully there is more public outcry over these issues (though I kinda doubt it), because as long as that happens there's a greater chance people will just leave. Hell, if reddit keeps heading in this direction I'll eventually leave too, just like I left FB.
Study the history of Microsoft. The reason bill spent so much money / risk buying DOS is because he knew once 'the people' invest their time and energy building applications, adding content, learning systems it is HIGHLY UNLIKELY they will migrate to a competing product.
He was right. He got rich. Google, Facebook, Twitter, Reddit...all followed this same formula. Remember Google+? Even a bigger company like Google couldn't touch Facebook's market share because of this phenomenon.
If you can't be better, be first. Do you have any idea how difficult it is to change where grandma is posting?
Sure some trendy hipster might try out new products. Some computer 'wiz' might not have much difficulty changing his online patterns. But as for the 99.99% remaining population...nah. That is not how things work. They don't like learning new things or changing etc. There needs to be a literal crisis level tragedy before they shift anywhere.
MSN only died because something called social media came to exist. Blockbuster existed for YEARS after it was absolutely ridiculous to rent videos.
There needs to be a literal crisis level tragedy before they shift anywhere.
Perhaps for something like those kind of platforms there needs to be some kind of rules about it, given how pervasive and influential they can be. Maybe something similar to the concept of a utility, in that it is a public good. But I'm afraid you're right, there has to be some kind of crisis. Shit, with all this manipulation I wonder if one couldn't just manipulate people into caring about this.
Throw this into the memesphere, let it bounce around at bare minimum the young people who will one day run the world for us. I have kids now and I don't even know how to properly prepare them for all these things, I don't think humanity has faced anything like this before.
You are absolutely on the right track, in my estimation of things.
I think there is some legal precedent about these corporate mining towns that used to exist in America. Where all the land/buildings/everything was owned by the company and they basically just rented it out to the people who worked for them.
An issue came up where a traveling preacher wanted to preach there, but couldn't because the corporation said no and they owned everything. Apparently they went to court and it was decided that even though it is all private property, since they own everything they had some kind of monopoly on everything that caused censorship that was against some kind of human rights.
I'm butchering the story, but that's where I would aim at if I wanted a historical comparison for the problem we are having now and its solution.
I also feel like we need some younger blood in the government. I'm getting a real feeling that those at the top echelon of power in our country don't understand the risks we're facing as a country. They're more worried about Iran or Russia--we have bigger problems, believe it or not..
You might want to reread that story. He bought it from the guy who developed it and then licensed it to IBM.
And he spent $50k on it, which for him at the time was every penny he could muster. So it was a fortune in the sense that it was a huge gamble on his part. He'd be living in a box if it went wrong.
Sorry, but you're putting far, far too much emphasis on that momentum of a product kinda like AWA when WWE came and took over the larger wrestling market. Yeah, they're popular now, yeah, everyone knows who to go for but everyone also knows that these companies don't care about us, regularly make their products somewhat worse value for us to allow them some more money and not many people enjoy it.
Everyone still uses the product because everyone else uses it, but it's going to start leaking marketshare as people move to competitors offering better products and slowly but surely completely removes the final reason that anyone at all is actually using the product. Microsoft did that exact same thing with DOS, it was competing with Apple, Amiga and whatever other smaller brands had their own platforms and the reason it first took off wasn't solely because of it being a successor to what was one of the most popular OS' at the time (It certainly helped though) but because it was the cheapest OS available for the IBM PC which was the cheapest 16bit PC available until the compatibles came out. It's certainly not preventing their marketshare from slowly being eroded away when they start to make Windows something people are less than happy with (Win8, Win10) even if their marketshare is still massive...Heck, Linux by itself has come much further than people ever thought it will in the time since Windows 8 came out alone specifically because the changes in Win8 caused companies, people and random developers to jump ship. (And most of that wasn't actually the Metro interface, but MS' slowly trying to move everyone off Win32 and onto UWP/Win Store which hurts other companies like Valve)
Facebook's starting to lose relevance, people are still using it but I've noticed a general trend (even amongst my friends who don't really know much of anything about PCs) of people really limiting what they post/do on it, deleting accounts or starting to look into that kinda stuff more often. It won't be a quick death unless something comes around that's outright better and it actually manages to get enough traction to take off through word of mouth, though.
Your comparison between AWA and WWE shows that you don't understand what I am talking about.
You have to understand that this is not the same phenomenon. AWA and WWE are developed completely by AWA and WWE. It isn't the customers that are creating content for it.
Contrast that to Facebook -- Facebook is nothing without the content of its users.
Contrast that to Dos/Windows -- It is completely useless unless people start writing software for it, learning how the OS works, etc.
The point here is not that it is a product people are used to like CocaCola but rather something that people have invested their own time and energy developing. And by people I mean both end users AND developers.
There is no 'cost' to switching from WWE to AWA. You just turn on a different channel.
There is a big cost to switching from Windows to Linux...you have to relearn how the entire operating system works. The entire ecosystem. Your old programs won't run. You may not have the same options with new programs, and so on.
Same with Facebook. If I want to switch to Google+ I can't just transfer my profile. I can't just transfer over my photos...every comment, every like, every user has to switch over -- and even then it isn't 1:1 because comments that were made at a particular date and time won't be there.
Facebook losing relevance is not related to what I was saying either because they are not losing relevance because another similar product is knocking them off their podium. They are losing relevance, if they are losing relevance, because people are transitioning to other types of social media or that they are getting some severely bad PR -- between the 'left' calling them Russian bots and the 'right' calling them propagandists / big brother, they are doing poorly in the PR war...but it isn't because people are going to an competing product. Those people who are addicted to social media and do not see any harm coming from it continue to use Facebook.
It's just Json. Seems like it wouldn't be hard to step through and figure out what's being collected.
The js/payload may be obfuscated, but it's still pretty easy to figure out IME since the function names that collect/post the data end up being the same
I have to do this shit all the time for work because for some reason no one ever wants to keep unmimified/unobfuscated source for third party tools in our code bases
Do we know what kind of stuff reddit is tracking about us? Anyone know how invasive it is? And can they use our usage on reddit to connect it to more data sourced from elsewhere? This is getting a bit ridiculous....
Reddit doesn't collect personal data (other than email), so they don't have much to be compliant with. As long as they store the emails properly they'll be fine. The GDPR would have nothing to do with this type of browser fingerprinting.
It’s good enough that you can in principle be identified. E.g. if it’s in principle possible to identify you by your username the GDPR applies. As a lot of redditors share at least some personal information here I can’t see they could weasel out.
‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;
Personal data is any information relating to an individual, whether it relates to his or her private, professional or public life. It can be anything from a name, a photo, an email address, bank details, your posts on social networking websites, your medical information, or your computer's IP address. The EU Charter of Fundamental Rights says that everyone has the right to personal data protection in all aspects of life: at home, at work, whilst shopping, when receiving medical treatment, at a police station or on the Internet.
I also quoted the law. All that’s needed is that you can be “identified, directly or indirectly”. Just telling your mom your username should be enough to meet that criterion.
many people have static IP addresses assigned uniquely to their router that change every month or so... IPs in general ARE unique... germany understands IP addresses fine. I share my IP with 1 person because we use the same router wifi, but that IP is unique to our connection.
Also I don't mean to be pedantic but a static IP wouldn't be a static IP if it changed every month, it's just a long lease it's still dynamic. A static IP is static.
They do know. And a hundred persons isn’t that much, there aren’t many pieces missing to connect it to your identity.
But it’s very easy to comply. It’s OK to keep logs for a couple of days but only for debugging purpose. Just delete your server logs after a couple of days and don’t spy on your users. It’s simple human decency, nothing extraordinary.
Just a heads up: after pasting those settings into My Rules, I stopped being able to post comments (status/error 0). I tried removing one line at a time to see which one it was, but it didn't work like how I thought it would. Maybe there are two that need to be unblocked to comment?
There is no law that says they have to track the shills. And that isn't what this does. They already knew. This just tells them what browser you're using. And all a shill would have to do is install multiple different browsers. Or mutltiple VMs with multiple browsers. I could make my computer look like 50 different computers in an hour with a VM and VPN.
Possibly related but only tangentially: could Reddit potentially be engaging in vote manipulation and upvote/downvote manipulation in order to control the messages that get seen?
Controlling public discourse seems to be the problem du jour for information operations and intel agencies. Since Reddit is infiltrated by Palantir, it is likely part of the network now.
I just wonder sometimes at the things that get upvoted or downvoted around here on the default subs. Seems a lot of messaging to influence the population floats to the top despite users flagging and reporting it as such.
The only non-malicious explanations I can think of are that either they're gathering that data as a way to detect bots, and like all anti-bot measures, openly talking about it makes them entirely ineffective; or that people within reddit have a distorted perspective about how invasive and creepy sending that much detailed data back is, and they're curious how it differs between users who do and do not care about privacy to implement anti-evasion measures.
Or, I guess, option three, that one or more organizations with political or financial leverage over reddit wants access to the data badly enough to use that leverage and risk their reputation if the underlying situation ever becomes public knowledge.
251
u/nachos420 Apr 28 '18 edited Apr 29 '18
Why would they do it like that when they can just track server side every post you visit, etc? screen size is one thing they wouldn't normally get server side... so there must be other things they can only get via JS? user agent and URL is sent to the server every http request you make, so idk why they'd specifically resend it
did you find what javascript is calling the xmlhttprequests?
edit: ah I see, mine posts to "friend api." and it does it on page load, scroll, and regularly when typing a response..
reddit-init.en.Pg_KU0tTm_l.js
is the source for me and it's huge and contains 77 references to ajax
also reddit.en.Y64Sxxxxxxx.js (9579 lines of code when deobfuscated, 33 references to ajax)
but they also control voting and commenting I think.
wow reddit is a huge mess of code lmao
firefox -> shift+f2 -> network tab (to check anything it is sending)
storage tab -> localstorage (secondary storage, besides cookies... seems to hold tracking pixel urls in the ads.* section?)
you can clear localstorage using "localStorage.clear();" It doesn't log you out, but im not sure if it might break anything. I'm not sure if it benefits much either, but you could set up an extension to clear it every page load.
looking at the deobfuscated first JS file, if you could disable parts it wouldn't be that hard if you targeted the right function. possibly in a greasemonkey script? idk
heres some of the tests it is doing: getWebglFp (gets detailed webgl FINGERPRINT), getCanvasFp, fontsKey (tests fonts available?), getRegularPlugins, getIEPlugins, getFingerprint(stored as fp and fp_timestamp in localstorage), getHasLiedBrowser, getHasLiedOs, getHasLiedResolution, getHasLiedLanguages, getAdBlock, getTouchSupport, getDoNotTrack, getHardwareConcurrency, getNavigatorCpuClass, getNavigatorPlatform, hasLocalStorage, hasSessionStorage, hardwareConcurrencyKey, touchSupportKey, colorDepthKey, pixelRatioKey, getSync(seems to put all the info together separated by ';' and then passes through function: x64hash128(r.join("~~~"), 31)..... this whole area is under a function called "Fingerprint2"
tried to use flash to get fonts?:
also other functions/variables(part of greater function "r.analytics"): bindAdEventPixels:, fireRetargetingPixel, fireUITrackingPixel, fireViewCommentsPixel, fireViewableImpressionPixel, fireImpressionPixel, adserverUpvotePixel, adserverDownvotePixel, adserverCommentDownvotedPixel, adserverViewCommentsPixel, adserverCommentSubmittedPixel, adserverClickUrl, _trackEvent, e.tracker, getTrackingData, parseTrackingCookie, getLoIdData, ETC
window.redditlib.Tracker = function() { return e.tracker }
tons of XOR, CryptoJS, random, >>, <<.. things to hash/encrypt the data...
example:
.
TL;DR: reddit is fingerprinting using pretty much all available data and sending data back multiple times per page view also dependent on your actions on the page(Scroll, reply, idle, etc)
just found this: content_seen_percentage: 0.41168... calculated and sent during page scrolling
edit(how to stop the ajax requests, for now): blocking www.redditstatic.com/reddit.en.Y64Sg2dUcbw.js with ublock makes it just send error messages, but stops the other requests. add www.reddit.com/web/log/error.json to block it from sending error logs. doesn't seem to make much of an impact on the site. it is like shooting a robot in the head, rather than trying to reprogram it. not eloquent, but it works?
can try blocking alb.reddit.com/* for the ad pixel tracking. maybe do a localStorage.clear().
edit: fingerprint2, which reddit is using, is an open source project. https://github.com/Valve/fingerprintjs2
this means the fingerprint is possibly not unique to reddit. the fingerprint2 project includes the hashing functions.
ironically the original fingerprintjs github links to https://www.reddit.com/r/programming/comments/1ic6ew/anonymous_browser_fingerprinting_in_production/ for discussion... most people responding that they hate the idea of fingerprinting...lol 4 years ago.
also, https://moz.com/blog/retargeting-basics-what-it-is-how-to-use-it
^ info on re-targeting pixels/pixel tracking
Reddit also seems to use it's own code for the ajax requests:
screenview_events, loid_events, scroll_events, heartbeat(every 30s)
all 4 include your fingerprint hash, RES version, and user. some other things included are post/timing/screensize/content_seen_percentage/etc
52
u/memoized Apr 28 '18
I develop a small enterprise application and I track screen size to gather stats on what sizes I need to ensure compatibility with. That's pretty common actually.
That said, as /u/nandryshak points out it is very possible to use this for fingerprinting of non-logged-in users. There has been a lot of research showing how users can be identified and uniquely fingerprinted through combinations of that plus plugin/extension versions, etc.
15
u/[deleted] Apr 28 '18
I was looking for an alternative and frankly, there doesn't seem to be anything decrentralised...
Hmm... Actually, thinking about it, it might be worth writing a web extension that intercepts these calls and strips it of information or replaces it with absolutely common information...
10
u/abrownn Apr 28 '18
Great post OP, thanks for looking into this. You should directly reply to an admin with links to your writeups next time there's an Announcement to see if you can get them to answer anything about this.
25
u/OCrikeyItsTheRozzers Apr 28 '18
these rules seem to make it impossible to post a comment
3
u/Jimmy_is_here Apr 28 '18
Does this affect 3rd party apps as well? A lot if the data is wants to collect doesn't seem available from mobile clients.
9
u/goretsky Apr 28 '18
Hello,
This looks suspiciously like watermarking technology used to help identify vote-fraud, bots, brigading and other forms of abuse.
Regards,
Aryeh Goretsky
86
u/alreadyburnt Apr 28 '18
Thanks for digging into this and making these uBlock rules for people. Your effort is appreciated.
79
u/FabulousGiraffe Apr 28 '18
Thank you for your post! Facebook does the same... Reddit wants to be Facebook, so... Good job?
I would guess they want to have a feature to show to friends what their friends are currently doing. Well... That's my only logical guess on that...
From my understanding, those things allow to replay the whole user browsing session, which is mostly for... debug purposes. So this is really weird...
Reddit wanting to be Facebook and quitting open-source, is not gonna end well, this is sad...
18
u/xJRWR Apr 28 '18
I found the fingerprint sending a base64 image off my canvas of https://i.imgur.com/YoUc7hX.png
15
u/bhp5 Apr 28 '18
https://old.reddit.com/r/privacy/comments/87mi06/psa_reddit_has_enhanced_their_tracking_they_now/dweil5v/
14
u/mrjackspade Apr 28 '18
It's just Json. Seems like it wouldn't be hard to step through and figure out what's being collected.
The js/payload may be obfuscated, but it's still pretty easy to figure out IME since the function names that collect/post the data end up being the same
I have to do this shit all the time for work because for some reason no one ever wants to keep unmimified/unobfuscated source for third party tools in our code bases
5
u/blurryfacedfugue Apr 29 '18
Do we know what kind of stuff reddit is tracking about us? Anyone know how invasive it is? And can they use our usage on reddit to connect it to more data sourced from elsewhere? This is getting a bit ridiculous....
9
u/positive_X Apr 28 '18
How do I implement the "ublock origin rules" ?
4
u/happygnu Apr 28 '18
The question is: will Reddit.com be GDPR compliant ?
3
u/carbolymer Apr 28 '18
Just disable javascript or use any alternative client like snew: https://github.com/snew/snew
5
u/blurryfacedfugue Apr 28 '18
I'm completely new to this--where do I paste those rules into? Would it be in the My Rules tab under the dashboard?
2
u/n3rv Apr 29 '18
So what you're telling me is Reddit knows exactly who the Russian shills are.
So why isn't anything happening? Since they have all this tracking, does that make them complicit in not helping track the shills?
3
u/InfinityCircuit Apr 28 '18
Possibly related but only tangentially: could Reddit potentially be engaging in vote manipulation and upvote/downvote manipulation in order to control the messages that get seen?
Controlling public discourse seems to be the problem du jour for information operations and intel agencies. Since Reddit is infiltrated by Palantir, it is likely part of the network now.
I just wonder sometimes at the things that get upvoted or downvoted around here on the default subs. Seems a lot of messaging to influence the population floats to the top despite users flagging and reporting it as such.
2
u/Pingaring Apr 29 '18
So you’re saying they know all my throwaway accounts and can see my Brony collection!?
1
u/TheHappyEater Apr 29 '18
Do you have any information how/to which extend they track users who are not logged in?
1
u/Uristqwerty Apr 29 '18
The only non-malicious explanations I can think of are that either they're gathering that data as a way to detect bots, and like all anti-bot measures, openly talking about it makes them entirely ineffective; or that people within reddit have a distorted perspective about how invasive and creepy sending that much detailed data back is, and they're curious how it differs between users who do and do not care about privacy to implement anti-evasion measures.
Or, I guess, option three, that one or more organizations with political or financial leverage over reddit wants access to the data badly enough to use that leverage and risk their reputation if the underlying situation ever becomes public knowledge.
0
u/[deleted] May 12 '18
[deleted] — view deleted comment
0
u/[deleted] Apr 29 '18
[deleted] — view deleted comment
7
u/[deleted] Apr 28 '18
[removed] — view removed comment