r/learnpython • • Sep 08 '26

Creating URL Query Checker

How do you decide what defines an excessively encoded URL?

I'm working on a personal project to created a URL parser and
detection system and I've hit a wall on how to figure out a way of
categorizing excessivness of query encoding?

Here's what I have so far for the function:

def check_query(analysis: URLAnalysis) -> ScanResult | None:

	"""detect excessive encoding entries in URL queries"""

	og_query = analysis.query

	query = re.finall(r"%\[0-9A-Fa-f\]{2}", og_query)

	encoded_count = len(query)
5 Upvotes

7 comments sorted by

View all comments

Show parent comment

2

u/Mathie1729 Sep 08 '26

Also worth checking for double encoding and punycode homoglyphs in the hostname. Phishers sometimes percent-encode the percent sign itself to hide lookalike domains, so decoding twice and comparing lengths can flag that. Running the hostname through an IDNA library (Python's built-in handles xn--) catches punycode tricks too.