Text Sorter & Deduplicator

Line-Based Text Operations: A Practical Toolkit

A lot of everyday work boils down to "I have a list of things; clean it up." Sort the lines. Remove duplicates. Strip blanks. Reverse the order. Each operation is trivial in shell (sort, uniq, awk, tac), but if you're not at a terminal, you reach for tools like this one. The point is speed: paste, click, copy, done.

Sort Variants

Alphabetical (A–Z / Z–A)

Lexicographic comparison by code point. ASCII uppercase comes before lowercase (A = 65, a = 97). For mixed-case input, this gives Apple, Banana, apple, banana which often isn't desired, use case-insensitive option to fold cases.

Locale-aware

For human-facing lists with accents, ligatures, or non-Latin scripts. JavaScript's Intl.Collator understands locale rules:

Not always the right call, for filenames or IDs, lexicographic is more predictable.

By length

Shortest-first or longest-first. Useful for:

• Finding outliers (suspiciously long log lines).
• Tightening up CSV columns.
• Identifying paragraphs that need editing.

Numeric

Treats each line as a number. 5, 10, 50, 100 instead of lexicographic 10, 100, 5, 50.

For lines like "100 items", picks up the leading number.

Natural sort

Numbers within strings sort numerically:

How users intuitively expect mixed alphanumeric to sort. macOS Finder uses natural; Linux ls doesn't (use ls -v).

Random shuffle

Fisher–Yates shuffle: pick a uniformly random permutation. Useful for randomizing test order, generating random samples, hiding hidden patterns in data.

Reverse (not really sort)

Reverse the existing order. Useful for "make this list end-first."

Deduplication

Set dedup (the common one)

Keeps first occurrence; drops all later duplicates. Equivalent to sort | uniq or Python set() (but preserving order).

Consecutive dedup

Drops only adjacent duplicates. Equivalent to Unix uniq without sort first.

Use for log compaction where you want to keep changes but collapse runs.

Case-insensitive dedup

Treats Apple and apple as the same. Use for human-entered lists; avoid for case-sensitive data (Unix paths, IDs).

Trim before dedup

Strips leading/trailing whitespace before comparing. "hello" and "hello " (trailing space) become identical. Almost always wanted.

Cleaning Operations

Remove blank lines

Drop lines that are empty or whitespace-only. "" and " " both removed.

Useful before processing, empty lines confuse downstream parsers and visually clutter output.

Trim each line

Strip leading and trailing whitespace from every line. Doesn't touch internal spaces.

For aggressive whitespace handling, also collapse internal multi-spaces.

Strip duplicate spaces

Replace runs of multiple spaces with single spaces. Useful when input has inconsistent spacing.

Add line numbers

Prefix each line with its 1-indexed line number. Useful for:

• Code review references ("line 42 ...").
• Creating numbered lists from plain text.
• Citing positions in long output.

Formats vary: 1., 1), 1:, or zero-padded 001 apple.

Common Workflows

Cleaning a copy-pasted list

You paste lines from an email or spreadsheet:

1. Trim each line.
2. Remove blank lines.
3. Deduplicate (case-insensitive).
4. Sort A–Z.

Result: clean, sorted, deduplicated list.

Preparing test data

You want N random items from a large list:

1. Shuffle.
2. Take first N (using the count display).

Random sample of any size.

Comparing two lists

To find items in list A not in list B:

1. Paste list A, deduplicate, sort.
2. Save output.
3. Paste list B, deduplicate, sort.
4. Use a diff tool to compare.

(For real set operations, a proper diff tool is faster.)

Reversing a log

Most logs go oldest-first; you want newest-first to read in browser without scrolling:

1. Reverse order.

Done.

Sort Stability

A stable sort preserves the original order of equal elements. JavaScript Array.prototype.sort is stable since ES2019. This matters when sorting by a partial key:

The tool uses native Array.sort (stable on all modern browsers).

Special Cases

Mixed-line-ending input

CRLF (Windows) vs LF (Unix) vs CR (old Mac). The tool splits on any line ending, then joins with the platform default. Round-trip preserves content but may change line endings.

Very large inputs

JavaScript can sort millions of lines, but the UI may stutter. For 10M+ lines, use CLI tools (sort, awk) or streaming pipelines.

Unicode

Sorting Unicode is locale-dependent. ASCII lexicographic is wrong for accented characters in most languages. Use Intl.Collator for language-aware sort.

Common Mistakes

Sorting before processing comments, sorting source code lines mixes comments with code unhelpfully. Sort only data lines, not structured content.

Deduplicating without trimming, 'foo' and 'foo ' both kept; trim first.

Case-sensitive dedup on human-entered data, Email and email both kept. Use case-insensitive option for fuzzy human input.

Sorting JSON / YAML lines, destroys the file. Only sort content where order doesn't matter.

Forgetting natural sort, file lists sort weirdly without it.

Privacy

All operations are JavaScript array methods running locally, split, sort, filter, Set, map, reverse. Open DevTools Network during use: zero outbound requests. Lines often contain internal IDs, usernames, error logs, customer data, or URLs, sending them to a server for "sorting" would be unnecessary disclosure. The tool runs in your browser so the text never leaves.