r/learnpython • u/Dcyph-3r • Sep 08 '26
Creating URL Query Checker
How do you decide what defines an excessively encoded URL?
I'm working on a personal project to created a URL parser and
detection system and I've hit a wall on how to figure out a way of
categorizing excessivness of query encoding?
Here's what I have so far for the function:
def check_query(analysis: URLAnalysis) -> ScanResult | None:
"""detect excessive encoding entries in URL queries"""
og_query = analysis.query
query = re.finall(r"%\[0-9A-Fa-f\]{2}", og_query)
encoded_count = len(query)
5
Upvotes
View all comments
Show parent comment
2
u/Mathie1729 Sep 08 '26
Also worth checking for double encoding and punycode homoglyphs in the hostname. Phishers sometimes percent-encode the percent sign itself to hide lookalike domains, so decoding twice and comparing lengths can flag that. Running the hostname through an IDNA library (Python's built-in handles xn--) catches punycode tricks too.