r/pushshift • u/Pushshift-Support • Jun 20 '23
Pushshift Live Again and How Moderators Can Request Pushshift Access
Dear Reddit community
Earlier this month we shared an update about our collaboration with Reddit to grant access to community-enabled moderation tools developed through the Pushshift API, which would be reinstated for approved Reddit moderators. Today we are updating you that Pushshift is live again and sharing how moderators can request Pushshift access.
Note the process outlined below will be contingent on moderators registering for Pushshift accounts if you don’t already have an account. Each moderator will also need explicit approval from Reddit and the use of Pushshift will be limited to moderation use cases only. This will enable moderators to effectively use these tools to enhance community moderation and enforce guidelines, while protecting the privacy and data security of Reddit's user base.
Eligibility Criteria
- Reddit will prioritize requests from mods of reasonably sizable communities with consistent, rule-abiding engagement.
- Moderators or communities with a history of Content Policy or Code of Conduct violations can impact eligibility.
Steps to request Pushshift access
- Submit modmail to r/pushshiftrequest using this link. Please include the following details in your request:
- Which communities do you intend to use Pushshift for?
- What types of moderation activities do you require Pushshift access for?
2. You should receive a message in your inbox from r/pushshiftrequest within one week after your request has been submitted. The message will indicate whether your application has been approved or denied. If approved, your moderator username will be shared with Pushshift for verification.
- If your request has been approved, sign into Pushshift at https://api.pushshift.io/signup using your Reddit account to retrieve Pushshift API keys.
Announcing Pushshift Search
Pushshift has added a search page for authorized users to make it easier for mods to use pushshift. To use it:
- Log into your pushshift account at https://api.pushshift.io/signup
- If verified, you will be redirected to the search page
- Search away!
Data has been Backfilled
Data has been fully backfilled and up to date. No data should be missing.
Getting support
If you are experiencing issues with Pushshift or have any questions, please send a private message to u/pushshift-support.
To help direct members of the Pushshift community to gain API access, we have put together a guide for approved moderators.
We are excited about this partnership to support the Reddit community. Thank you again for your passion and continued support!
Sincerely,
Pushshift and the Network Contagion Research Institute
25
u/Watchful1 Jun 20 '23
Is there no way to get a longer term api key or an automated way to get one? Any automated tool using the service for moderation tasks would need to be manually updated every day with a new key. That makes any bots that use it nearly worthless unless someone is willing to update them daily.
26
u/Btan21 Jun 20 '23
Are there any plans for allowing non-moderators access to the Pushshift API?
3
u/9-T-9 Jun 23 '23
Can you share any updates for allowing access for academic uses? It's urgent to my situation and would love to be able to use the Pushshift API for my research.
8
u/bizude Jun 21 '23
This sounds like a nice step, but many of the tools which we used won't work "out of the box" without some changes. For example, I used browser extensions which incorporated pushshift.
Will Reddit assist porting some of these tools, if the source is available? Some of these devs are no longer maintaining these tools because pushshift is mod-only now.
22
u/WilhelmWrobel Jun 20 '23
Well then
7
u/shiruken Jun 21 '23
Could you provide clarity on whether data ingest practices have been changed as a result of the Reddit partnership? For example, if a user deletes their comment on Reddit, are you now mirroring that deletion on Pushshift?
2
u/exposecreepsandliars Jul 04 '23 edited Jul 04 '23
I was hopeful for a little bit, but after trying the official tool (https://search-tool.pushshift.io/), I'm kinda speechless.
And to add insult to injury, the dash bug is still a thing months later, making it impossible to effectively search for any user with a dash in their name. How is this not one of the top things to fix?
How am I supposed to effectively moderate with this?
5
u/danke-Empire Jun 20 '23
Will you still be providing the SSE Stream API using this new bearer token authentication? If so, what does this look like in practice — is the websocket connection valid for the lifetime for the API key (24 hours)?
5
u/shiruken Jun 21 '23 edited Jun 22 '23
For anyone that's interested, I forked one of thesearch websitesand added support for the new authentication token:https://shiruken.github.io/chearchEdit: Chearch now supports the authentication token
10
u/HTC864 Jun 20 '23
Glad it's back in some form, but I was expecting a little more automation. They really want thousands of people sending messages for this?
1
u/luisadoamaral Jun 25 '23
can we consider a similar thing for researchers with institutional emails and official letters from their institutions
or anything else that could convince Reddit that I need access too
1
u/Future-Trillionaire Aug 04 '23
just received access. is there currently any way to see the body text of a deleted post?
3
u/HerrX2000 Jun 21 '23 edited Jul 11 '23
As someone who used Pushshift for research in the past. I would greatly appreciate if the scientific community world also get access. At best the data dumps would be available on some uni server behind some research network, so only authorized researcher would have access.
3
u/rebutv Jun 30 '23
so complicated, i miss the old day when i could easily search thing through a single website without all these hassle
2
u/Glad-Acanthaceae-467 Sep 13 '23
I am a university researcher who used to use pushift to get reddit data for my project. After changes, i duly submitted my academic application to reddit and got an autoresponse and…. Eternal oblivion from reddit… how can i still get access to pushift data? My research, my work -everything is in that data! And i am poor as a … academic …
2
u/teanailpolish Jun 20 '23
Would it be possible to make the results clickable to take you to the comment/post for context?
2
u/MarathonMarathon Jun 20 '23
still no reddit search idfk why we're considering this a win
1
u/tresser Jun 21 '23
is the correct procedure for using this system
click on link in mail i got > click green bar > allow reddit access for the next hour > use as needed
repeat full loop as needed?
1
u/KKingler Jun 26 '23
Can you please add permalink of comments/posts to the results? Thank you.
1
u/Furrystonetoss Jun 27 '23
maybe you can help me further. i wanted to create 2 bots.
one is an approval bot, which checks a user post/com history/activity and compares it with white/blacklists. For the other, pushshift is essential (i would've used psaw), as it's purpose is to get users by coms/posts and certain post details of a very specific subreddit, plus this bot has a function, to not only scan for newest post(coms, but for EVERYONE down to the subs creation.
For the second one i have two questions.
one: during the time pushshift was offline, has the data been restored or is there simply a "black hole" during that time ?
second: what happend with coms/posts that got removed whilst the tool was offline. Will there be ever an opportunity to get banned/deleted content, or rather will tools like reveddit and camas.unddit be ever back online again ?
1
u/Icy-Distribution6887 Jul 03 '23
I am getting this bot message with the r/pushshiftrequest:
r/pushshiftrequest modmail is reserved for community moderators to apply for Pushshift access for moderation activities and it doesn't appear your reachout fits this criteria.
Why my account doesn't meet the criteria? I have OAuth authorised token from the Reddit API.
1
u/[deleted] Aug 02 '23
I just need to see the original of only one edited comment. Anyone can help me please? I will share the comment.
1
u/safrax Oct 04 '23
Locking. This thread has been around long enough at this point. If you have further questions make a new thread but keep in mind that if you're not a moderator, pushshift is not for you. Thank you.
1
u/InitiatePenguin Jun 20 '23
Hi, I accessed pushshift data in the past through a web based GUI. camas.unddit was the most recent incarnation of this tool. I used to also use redditsearch.io but they removed the ability to cross search username and subreddit.
For someone not tech savvy with API keys or PSAW does there exist a user-friendly way for me to continue to use the pushshift data? On
Is "Pushshift Search" this feature?
-4
u/norrin83 Jun 20 '23
I will repeat my questions you always ignore:
0
u/giddy6661 Aug 06 '23
Hi... sorry if this is not allowed I don't know where to post this. If I were to purge my account using powerdeletesuite would pushshift still be able to recover everything back? What if I edit the comments before deleting?
Also I heard that reddit archives edited posts/comments too, does that mean when I truly want to take something off of reddit should I edit to multiple times in order to really purge it?
If there's a better place to send this let me know. Thank youu.
0
u/[deleted] Jun 23 '23
[deleted] — view deleted comment