Kodeclik Blog
3 Simple Ways to Remove Duplicates from a Javascript Array
Duplicate values in arrays are a common challenge when working with Javascript data. Whether you're processing API responses, handling user input, or merging data from multiple sources, you'll often need to ensure each value appears only once.
Here are 3 common ways to remove duplicates from a Javascript array:
Method 1: Use Set
The simplest and most performant approach is to leverage Javascript's Set data structure, which automatically stores only unique values.
const numbers = [1, 2, 2, 3, 4, 4, 5];
const unique = [...new Set(numbers)];
console.log(unique); // [1, 2, 3, 4, 5]
This code creates a new Set from the numbers array, which immediately removes all duplicate values since Sets can only contain unique elements. The spread operator then converts the Set back into a standard array. This method works efficiently for primitive values like numbers, strings, and booleans, and is generally the preferred approach for most deduplication tasks due to its clean syntax and optimal time complexity.
Method 2: Use filter() + indexOf()
The filter method combined with indexOf provides a more traditional approach that works well when you need finer control over the deduplication process.
const items = ['apple', 'banana', 'apple', 'orange'];
const uniqueItems = items.filter((item, index) =>
items.indexOf(item) === index
);
console.log(uniqueItems); // ['apple', 'banana', 'orange']This technique filters the array by keeping only elements whose first occurrence (found by indexOf) matches their current position in the array. When filter encounters the second 'apple' at index 2, indexOf returns 0 (the first occurrence), so the condition fails and that duplicate is excluded. While this method is more verbose than using Set, it offers better browser compatibility with older Javascript environments and can be easily modified to include custom comparison logic.
Method 3: Use a Hash Map (reduce)
The reduce method provides a flexible approach that builds a new array by accumulating only unique values.
const arr = ['a', 'b', 'a', 'c', 'b'];
const uniqueArr = arr.reduce((acc, item) => {
if (!acc.includes(item)) acc.push(item);
return acc;
}, []);
console.log(uniqueArr); // ['a', 'b', 'c']This code uses reduce to iterate through the array while maintaining an accumulator that starts as an empty array. For each element, it checks whether the accumulator already contains that value, and only adds it if it's not present. Although this approach has a higher time complexity than Set due to the includes check on each iteration, it demonstrates the underlying logic of deduplication clearly and can be adapted for more complex scenarios where you need to perform additional operations during the deduplication process.
Deduping Objects
If you have an array of objects, you can remove duplicates by a key:
const users = [
{ id: 1, name: 'Ava' },
{ id: 2, name: 'Ben' },
{ id: 1, name: 'Ava' }
];
const uniqueUsers = [...new Map(users.map(u => [u.id, u])).values()];
console.log(uniqueUsers);
// [{ id: 1, name: 'Ava' }, { id: 2, name: 'Ben' }]This elegant solution uses a Map to deduplicate objects based on a specific property. The code first transforms the users array into an array of key-value pairs where each user's id becomes the key and the entire user object becomes the value.
When these pairs are passed to the Map constructor, any duplicate ids automatically overwrite previous entries since Map keys must be unique.
Finally, the values method extracts just the user objects, and the spread operator converts them back into a standard array. This technique is particularly useful when working with datasets where objects should be considered duplicates based on a unique identifier rather than complete structural equality.
Applications
Removing duplicates from arrays is a fundamental operation in many real-world Javascript applications.
When processing data from APIs or databases, you often encounter duplicate entries that need to be cleaned before display or further processing. For instance, when building a search feature that combines results from multiple sources, deduplication ensures users see each result only once.
Form validation and user input processing also benefit significantly from deduplication. When users submit tags, categories, or multiple selections, removing duplicates prevents data inconsistencies and improves the user experience.
Summary
Javascript offers several robust methods for removing duplicates from arrays, each suited to different scenarios.
The Set approach provides the cleanest and most performant solution for primitive values, while the filter and reduce methods offer more flexibility for complex logic and better compatibility with legacy browsers. When working with objects, using a Map with a unique key provides an elegant way to deduplicate based on specific properties.
Choosing the right deduplication method depends on your specific requirements including browser support needs, data types, performance constraints, and whether you need to preserve order or apply custom comparison logic.
Want to learn Javascript with us? Sign up for 1:1 or small group classes.
