Pin urlencode doseq and quote_via Before One Query Extract

Pin the query-string contract before any helper extract. Extract one encoder only after those bytes stay frozen. Messy HTTP clients scatter urlencode calls across modules. Each site may pass different flags without comment. A later extract then changes production query bytes. This article treats that scatter as a test problem. You freeze observed encodings first, then move one function. Why this refactor fails closed urlencode looks like a trivial one-line wrapper. Default flags still decide the exact wire bytes. The default doseq flag is False in urllib.parse. A list value then stringifies with Python brackets. Spaces become plus signs under the quote_plus helper. None becomes the literal four-character string None. Those outcomes are language facts, not style choices. Read them in the official urllib.parse documentation. Decision table: what to pin Pin every flag that can change wire format. Skip flags that this module never reaches. Input and flag Typical observed bytes Breakage if you flip it list value, doseq False tags=%5B%27a%27%2C+%27b%27%5D server sees one Python-list string list value, doseq True tags=a&tags=b duplicate keys appear on the wire space, quote_plus q=hello+world some APIs reject plus-as-space space, quote q=hello%20world other APIs reject percent-twenty None value x=None backends store the word None bool True debug=True backends expected 1 or true blank value, parse_qs default key disappears empty filters vanish on round-trip blank value, keep_blank_values q maps to [""] empty filters survive dict key order, 3.7+ insertion order signed query checks fail Do not treat the table as a benchmark run. Treat it as a checklist for one module. Step 1: Inventory every encode call Search the messy tree with one ripgrep command. Limit the search to Python files on purpose. rg -n "urlencode|urlunparse|parse_qs|parse_qsl|quote_plus|quote_from_bytes" -g "*.py" Enter fullscreen mode Exit fullscreen mode Record file, line, and explicit keyword arguments. Record missing keywords as implicit library defaults. Three columns are enough for this inventory. Use path, call, and inferred flags only. Stop after one module and refuse the extract. Step 2: Capture live query bytes You need observed strings, not imagined encodings. Add a temporary probe next to the client. # Proposed probe. Unexecuted instrumentation, not a library. import json import pathlib import urllib.parse PROBE = pathlib.Path("artifacts/query_probe.jsonl") def probe_urlencode(params, **kwargs): raw = urllib.parse.urlencode(params, **kwargs) rec = { "params": repr(params), "kwargs": sorted(kwargs.items()), "raw": raw, } PROBE.parent.mkdir(parents=True, exist_ok=True) with PROBE.open("a", encoding="utf-8") as fh: fh.write(json.dumps(rec) + "\n") return raw Enter fullscreen mode Exit fullscreen mode Route one client path through the probe helper. Hit the real fixtures you already keep. Do not invent extra synthetic production traffic. Label this probe as temporary test instrumentation. Delete it after the pytest suite exists. Step 3: Turn probes into characterization tests Move frozen rows into a pytest module next. Do not pretty-print the expected query strings. # tests/test_query_contract.py # Proposed characterization suite. Unexecuted in this article. import urllib.parse import pytest QUOTE_PLUS = urllib.parse.quote_plus QUOTE = urllib.parse.quote CASES = [ ( "doseq_false_list", {"tags": ["a", "b"]}, {"doseq": False}, "tags=%5B%27a%27%2C+%27b%27%5D", ), ( "doseq_true_list", {"tags": ["a", "b"]}, {"doseq": True}, "tags=a&tags=b", ), ( "space_plus", {"q": "hello world"}, {"quote_via": QUOTE_PLUS}, "q=hello+world", ), ( "space_percent", {"q": "hello world"}, {"quote_via": QUOTE}, "q=hello%20world", ), ( "none_literal", {"x": None}, {}, "x=None", ), ( "bool_true", {"debug": True}, {}, "debug=True", ), ( "empty_value", {"q": ""}, {}, "q=", ), ] @pytest.mark.parametrize("name,params,kwargs,expected", CASES) def test_urlencode_contract(name, params, kwargs, expected): got = urllib.parse.urlencode(params, **kwargs) assert got == expected, name Enter fullscreen mode Exit fullscreen mode Run the suite with one boring pytest command. A red test means the expected row is wrong. python -m pytest tests/test_query_contract.py -q Enter fullscreen mode Exit fullscreen mode Fix the row from the probe file. Do not fix it from personal taste. Copy expected strings from artifacts, not from memory. One mistyped percent sequence silently locks a lie. Step 4: Pin the inverse parse path Encoding is only half of the contract. Parsing the same bytes is the other half. The parse_qs helper drops blank values by default. The keep_blank_values flag keeps a bare equals. def test_parse_qs_keeps_blanks_when_module_does(): got = urllib.parse.parse_qs("q=", keep_blank_values=True) assert got == {"q": [""]} def test_parse_qs_drops_blanks_by_default(): got = urllib.parse.parse_qs("q=") assert got == {} Enter fullscreen mode Exit fullscreen mode Add inverse rows only if the module parses callbacks. Pick one blank-value behavior for the helper. Do not mix both behaviors inside one function. strict_parsing is a separate switch. Pin it only when the module already sets it. Step 5: Diff two call sites before merging them Two call sites may disagree on doseq today. Do not hide that conflict inside one helper. Write a comparison script before any merge. Print both encodings for the same input dict. # Proposed local comparator. Unexecuted here. from urllib.parse import urlencode, quote, quote_plus PARAMS = {"tags": ["a", "b"], "q": "hello world", "x": None} left = urlencode(PARAMS, doseq=False, quote_via=quote_plus) right = urlencode(PARAMS, doseq=True, quote_via=quote) print(left) print(right) print("match" if left == right else "conflict") Enter fullscreen mode Exit fullscreen mode If the two strings differ, stop the extract. Split helpers or pick a documented winner first. A helper that averages two wire formats is a defect. Characterization tests should make that conflict loud. Step 6: Extract the smallest encoder The suite is now the wire contract. Move one function and leave every other site. # client/query.py # Proposed extract. Keep flags literal. from urllib.parse import quote_plus, urlencode def encode_query(params): """Wire format pinned by tests/test_query_contract.py.""" return urlencode(params, doseq=True, quote_via=quote_plus, safe="") Enter fullscreen mode Exit fullscreen mode Replace one call site, then re-run the suite. Replace the next site only while tests stay green. Do not add key sorting in the helper. Do not coerce None into omitted keys. Do not lowercase boolean True into true. Those edits are new behavior, not an extract. Ship them later as a documented change. Keep the flags as literals in the helper. Indirection here hides the contract again. Step 7: Mutate on purpose A green suite can still be tautological. Flip one flag inside the new helper. # Temporary mutation. Do not commit this line. return urlencode(params, doseq=False, quote_via=quote_plus, safe="") Enter fullscreen mode Exit fullscreen mode The doseq characterization test must fail then. If it does not, list values never entered the suite. Restore the pinned flags after the mutation. Keep the failing experiment out of version control. Repeat once for quote_via if spaces appear. One red test per flag is enough evidence. Unicode, encoding, and the safe alphabet Non-ASCII values depend on the encoding argument. The library default encoding value is utf-8. Errors default to strict and can raise. Pin both kwargs if the module sets them. def test_utf8_city_value(): got = urllib.parse.urlencode({"city": "München"}) assert got == "city=M%C3%BCnchen" Enter fullscreen mode Exit fullscreen mode The safe alphabet also changes bytes. A slash left unescaped is a contract choice. Pin safe only when a call site already passes it. Do not invent a richer safe set during extract. Signed queries are a harder contract Signed query strings add a canonical-byte requirement. Insertion order then becomes part of the contract. Pin key order with explicit item lists, not dicts. def test_signed_pair_order(): pairs = [("b", "2"), ("a", "1")] got = urllib.parse.urlencode(pairs) assert got == "b=2&a=1" Enter fullscreen mode Exit fullscreen mode Do not let a model reorder signed parameters. A changed order is a product incident. If a signer also percent-encodes again, pin that too. Double encoding is common in messy clients. Where a model run fits Local tests remain the gate for this extract. A model does not replace the probe file. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode provides free model access and a free server option. You can send the frozen suite plus one module there. Accept a diff only when pytest stays green. Skip the remote run if the suite already guides you. If you use it, start from this suite, not from a prompt. Limitations This method does not define a public API. It freezes whatever the messy module already emits. Do not use it to tidy None or boolean values. That change is behavior, not structure. Do not use it for JSON request bodies. Do not use it for multipart form payloads. Do not use it for GraphQL variable maps. Python 3.7+ dict order is insertion order. Older runtimes are out of scope here. Nested dicts are also out of scope. urlencode will stringify them with Python reprs. Bytes keys and values need a separate pin. This suite assumes text params on purpose. Who should skip this Skip this if you own no query-building code. Skip this if every call already shares one helper. Skip this if a written HTTP contract already names flags. Skip this for binary protocols. Skip this when legal or billing queries are unsigned still. Wait, skip those until a signer and byte order exist. Skip this during an incident hotfix. Characterization belongs in a quiet change window. Checklist Inventory urlencode and parse_qs call sites. Probe real fixtures into a jsonl file. Promote rows to pytest equality checks. Diff disagreeing call sites before any merge. Pin parse blanks if the module round-trips. Extract one helper with literal flags. Mutate one flag and confirm a red test. Restore flags and delete the probe. Query bytes are the product you ship. Helpers are optional once those bytes stay frozen.