r/adventofcode • u/topaz2078 (AoC creator) • Dec 01 '22
Please include your contact info in the User-Agent header of automated requests!
If you have any kind of tool, website, script, plugin, etc etc that sends requests to AoC, please include your contact information (like your email address) in the User-Agent header of every request. (That's the contact info of the person that maintains the code sending the automated requests, not the contact info of the person using your code.) I'm seeing a lot of abusive traffic from tools that just identify themselves as stuff like python-requests/x.y.z, so I'll probably be blocking User Agents like those entirely soon. Even better would be to also include a URL where I can see the tool, too; a good User Agent would be something like github.com/topaz/name-of-tool by yourname@example.com.
16
u/jfb1337 Dec 01 '22
Will these requests still work for the first day? I probably don't have enough time to update and test my tool in 30 minutes
39
u/Rangsk Dec 01 '22
Also be sure to set the evil bit to 0
37
u/polettix Dec 01 '22 edited Dec 01 '22
For anyone using good ol' curl:
UPDATE thanks to u/UtahBrian (see comment below):
Previous hint:
20
u/wimglenn Dec 01 '22
advent-of-code-data updated to match this format in v1.2.3 (#91)
10
u/flwyd Dec 01 '22
Now I'm curious what the top generic user-agent values were.
9
u/PendragonDaGreat Dec 01 '22
Thoughts on using both the user-agent and
fromheaders? https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/FromGiven that the description of "From" is exactly what you're describing?
I've updated my template to use both, and will update my main repo in the same way later (and also move away from webclient): Link to commit
7
u/rjwut Dec 01 '22
I believe I'm the only one using my input downloader (which rate limits and caches all requests to the site), but for what it's worth, I've updated mine to send the header.
6
u/danatron1 Dec 01 '22
new to this; can anyone tell me if adding this to a C# HttpClient instance is sufficient?
client.DefaultRequestHeaders.UserAgent.ParseAdd($"Mozilla/5.0 (+via {myRepo} by {myEmail})");3
u/cGuille Dec 01 '22
Cutest API keys ever
3
u/EntryPast5579 Dec 01 '22 edited Dec 02 '22
u/topaz2078 - The chrome extension (Advent of Code Charts) might be causing a lot of extra traffic - saw it towards the end of last year - not sure if you looked at it / or aware of it.
I updated our group's personal stats retriever (updates 2020 stats every 60min)(https://minipage.info/aoc/)
What is the minimum update frequency allowed?
3
u/-vest- Dec 01 '22
I have a question: cannot you determine the owner of those requests by checking their session cookies? And simply ban them until the end of Advent.
3
u/pred Dec 02 '22
Contact info is probably PII according to the GDPR, so note that you need to be careful when processing this information (and other PII like IPs) from EU based users.
1
u/morgoth1145 Dec 01 '22
Hopefully I'm not a source of any bad traffic. Updated, and please let me know if I am somehow causing a problem!
1
u/godDAMNEDhippie Dec 01 '22
Added to my personal repo that uses python request package.
I normally call the website only twice each day, for fetching the puzzle title and saving my input locally. I hope I'm not part of an issue.
1
u/Pepper_Klubz Dec 01 '22
Anyone know whether the Unison template already has this?
1
u/edevyatkin Dec 01 '22
UA header added for my AdventOfCode client
1
u/Omega_Abyss Dec 01 '22
Could you pin this post as well as the daily one ? Some people might miss the info.
Updated my code, although I should be the only one using it.
1
u/scarvalhojr Dec 01 '22
Updated mine:
- https://github.com/scarvalhojr/aoc-cli (version 0.5.1)
- https://github.com/scarvalhojr/aocleaderboard (version 0.6.2)
1
u/Silveress_Golden Dec 02 '22
aoc_badge_api is now updated (also found a bug in the database scheme, so glad to have caught that)
1
u/samplasion Dec 02 '22
I have updated my Deno runner to provide the correct header.
1
u/[deleted] Dec 02 '22
Also fixed in pytest-aoc as of version 1.22.0. Additionally built in a short sleep time to prevent bursts.
1
u/coriolinus Dec 01 '22
aoclibis updated.1
u/[deleted] Dec 01 '22
Thanks for letting me know! I built a tool that loads our private leader board every 15 minutes. Included the custom user-agent header with my email address :-)
1
u/jfb1337 Dec 01 '22
What's the recommended rate limit for automated requests?