Array Deduplication

Removes duplicates from an array of numbers or strings

Problem Overview

🔄 Remove the Noise, Keep the Signal
Array deduplication is a fundamental data cleaning operation that removes duplicate values while preserving the original order. Essential for data processing pipelines!

Why Order Matters:
Preserves chronological sequence in time-series data
Maintains user input order in form processing
Keeps first occurrence, discards subsequent duplicates
Critical for data integrity in processing workflows
Three Deduplication Strategies:
Set-Based: Uses Set for O(1) lookups - fastest approach
Filter + indexOf: Simple but O(n²) - good for learning
Reduce + includes: Functional approach - readable but slower
Real-World Applications:
User Input: Remove duplicate tags, categories, or selections
Data Processing: Clean imported datasets and APIs
Search Results: Eliminate duplicate entries from multiple sources
Analytics: Unique visitor tracking and event deduplication

💡 Learning Value:

Set vs Array performance characteristics
Time complexity analysis and optimization
Functional vs imperative programming styles
Data structure selection for specific use cases

Concepts

array manipulationhash setsfilteringreducing

Complexity Analysis

Time:O(n) for Set-based, O(n²) for filter/reduce-based

Space:O(n)

Solutions

ArrayDeduplicate

Time: O(n) | Space: O(n)

export function ArrayDeduplicate(arr: (number | string)[]): (number | string)[] {
  if (!Array.isArray(arr)) throw new Error("Input must be an array");
  const seen = new Set();
  const dedupedArray: (number | string)[] = [];
  for (const item of arr) {
    if (!seen.has(item)) {
      seen.add(item);
      dedupedArray.push(item);
    }
  }
  return dedupedArray;
}

Examples & Test Cases

#1Duplicate numbers

Input:

[
  1,
  2,
  2,
  3,
  1
]

Output:

[
  1,
  2,
  3
]

#2Duplicate strings

Input:

[
  "a",
  "b",
  "a",
  "c",
  "b"
]

Output:

[
  "a",
  "b",
  "c"
]

#3Empty array

Input:[]

Output:[]

#4Single element

Input:

[
  1
]

Output:

[
  1
]

#5Mixed types

Input:

[
  1,
  "a",
  1,
  "a",
  2
]

Output:

[
  1,
  "a",
  2
]

#6Invalid input

Input:null

Output:{}