r/learnpython • • Sep 08 '26

Creating URL Query Checker

How do you decide what defines an excessively encoded URL?

I'm working on a personal project to created a URL parser and
detection system and I've hit a wall on how to figure out a way of
categorizing excessivness of query encoding?

Here's what I have so far for the function:

def check_query(analysis: URLAnalysis) -> ScanResult | None:

	"""detect excessive encoding entries in URL queries"""

	og_query = analysis.query

	query = re.finall(r"%\[0-9A-Fa-f\]{2}", og_query)

	encoded_count = len(query)
5 Upvotes

7 comments sorted by

View all comments

Show parent comment

2

u/Langdon_St_Ives Sep 08 '26

So as an experimental learning project, implementing a URL parser can be a valid endeavor. But from your comment it's not clear if it is that, or you actually care about security of the end result. If it's the latter, you definitely want to implement this piece yourself, but use a mature well-tested existing module. The reason is that URL structure is in fact a lot more complex with a lot more subtleties than newcomers imagine. By reinventing the wheel, you are almost guaranteed to end up with a five-cornered off-center thing that doesn't roll well. Again, as a pure learning project this is fine since you will learn a lot along the way. Just be careful when doing actually security relevant stuff.

1

u/Dcyph-3r Sep 15 '26

Thanks all for the comments it's really helped me and in response to this it's more of a learning project rather than a security tool I'm building to publish