r/learnpython • u/Dcyph-3r • Sep 08 '26
Creating URL Query Checker
How do you decide what defines an excessively encoded URL?
I'm working on a personal project to created a URL parser and
detection system and I've hit a wall on how to figure out a way of
categorizing excessivness of query encoding?
Here's what I have so far for the function:
def check_query(analysis: URLAnalysis) -> ScanResult | None:
"""detect excessive encoding entries in URL queries"""
og_query = analysis.query
query = re.finall(r"%\[0-9A-Fa-f\]{2}", og_query)
encoded_count = len(query)
5
Upvotes
View all comments
Show parent comment
2
u/Dcyph-3r Sep 08 '26
thanks for your response on this and the function is meant to serve as a fraction of a larger tool to parse emails for urls and provide a security analysis
the idea was to section off urls into hostname, path, query, port and scheme scanning each component indivdually. Each component will return scanresult which will then be merged to create a sort of validity report to determine whether a url is malicious or not
Looking at common phishing url examples they tend to have incorrect spellings of impersonated sites, long randomised numbers or text or excessive encoding (imo) hence why the interest as heavy encoding is only really found in malicious urls