# Requirement: Post Text "Car Wash" Sanitization ## Context When a user pastes a social media link, the app fetches the post text to use as a label for that URL. This scraped text frequently arrives with encoding artifacts from the web — percent-encoded characters, HTML entities, Unicode escapes, and other noise that makes the label unreadable. **Example of what comes in:** ``` Don%E2%80%99t miss%20this%21 &amp; more – today%20%F0%9F%94%A5 ``` **Example of what the user should see:** ``` Don't miss this! & more – today 🔥 ``` --- ## Requirement **When the app retrieves post text from a social media URL, pass it through the `carWash()` function before using it anywhere in the UI** — as a label, preview, title, or stored value. ```js const cleanLabel = carWash(scrapedPostText); ``` ### What `carWash()` does The function runs a multi-pass cleaning pipeline in this order: 1. **URL percent-decode** — converts `%20` → space, `%E2%80%99` → `'`, etc. 2. **HTML entity decode** — converts `&` → `&`, `–` → `–`, ` ` → space, etc. 3. Repeats steps 1–2 in a loop until the output stops changing (handles stacked encoding like `%26amp%3B` → `&` → `&`), capped at 8 passes. 4. **Unicode escape decode** — converts `\u0041` → `A` 5. **Hex escape decode** — converts `\x41` → `A` 6. **Strip pipe characters** — removes any `|` characters 7. **Normalize whitespace** — collapses multiple spaces/newlines to a single space ### Where to apply it Apply `carWash()` immediately after fetching/scraping the post text, **before** it is stored, displayed, or used as a label. Do not display raw scraped text anywhere in the UI. ```js // Example integration point async function fetchPostLabel(url) { const raw = await scrapePostText(url); // however you currently fetch it return carWash(raw); // <-- add this line } ``` ### The function Drop `carWash()` into a shared utility file (e.g. `utils/carWash.js`) and import it wherever post text is handled. > ⚠️ `carWash()` uses `document.createElement` for HTML entity decoding, > so it runs in a browser context. If post fetching happens server-side > (Node.js), replace the HTML entity step with a library like > [`he`](https://www.npmjs.com/package/he): `he.decode(text)`. ```js /** * carWash(rawString) → string * * Cleans encoded or garbled text (from social media scraping, API responses, * or URL parameters) into plain human-readable text. * * Handles: URL percent-encoding, HTML entities (named + numeric), * Unicode escapes (\u0041), Hex escapes (\x41), and pipe characters. * * Usage: * const clean = carWash(scrapedPostText); * * Example: * carWash("Don%E2%80%99t miss%20this%21 &amp; more – today") * → "Don't miss this! & more – today" */ function carWash(raw) { if (!raw || typeof raw !== 'string') return ''; let text = raw.trim(); const MAX_PASSES = 8; let passes = 0; // Iterative URL + HTML unwrap loop — runs until nothing changes. // Needed because layers can stack: %26amp%3B → & → & let changed = true; while (changed && passes < MAX_PASSES) { changed = false; passes++; // URL percent-decode if (/%[0-9A-Fa-f]{2}/.test(text)) { try { const next = decodeURIComponent(text); if (next !== text) { text = next; changed = true; continue; } } catch (_) {} } // HTML entity decode (named: & and numeric: – –) if (/&[A-Za-z]+;|&#\d+;|&#x[0-9a-fA-F]+;/.test(text)) { const el = document.createElement('div'); el.innerHTML = text; const next = el.textContent; if (next !== text) { text = next; changed = true; continue; } } } // Unicode escape sequences: \u0041 → A text = text.replace(/\\u([0-9a-fA-F]{4})/gi, (_, c) => String.fromCharCode(parseInt(c, 16))); // Hex escape sequences: \x41 → A text = text.replace(/\\x([0-9a-fA-F]{2})/gi, (_, c) => String.fromCharCode(parseInt(c, 16))); // Strip pipe characters text = text.replace(/\|/g, ''); // Collapse multiple spaces/newlines into single space text = text.replace(/\s+/g, ' ').trim(); return text; } ``` --- ## Acceptance Criteria - [ ] `carWash()` is added to the project as a shared utility - [ ] All scraped post text passes through `carWash()` before display or storage - [ ] Labels never show raw `%xx`, `&`, `&#xxxx;`, `\uXXXX`, or `|` characters - [ ] If input is empty or null, the function returns an empty string gracefully