JavaScript Arrays: Removing Duplicates

Arrays in JavaScript are often used to store collections of data. However, when working with arrays, it’s common to encounter duplicate values. This can cause issues, especially when the data needs to be unique, such as when cleaning up data or ensuring that only distinct items are present in the array.

Removing duplicates from an array is a common task in programming. Whether you’re processing user input, working with a dataset, or ensuring uniqueness in an application, it’s essential to know how to handle duplicates efficiently.

In this article, we’ll explore different methods to remove duplicates from arrays in JavaScript. Each method offers unique benefits depending on the scenario, and we’ll cover a variety of approaches that you can use to suit your needs.

Table of Contents

Using `Set` to Remove Duplicates

One of the easiest ways to remove duplicates from an array in JavaScript is by using a Set. A Set is a built-in JavaScript object that automatically stores only unique values. When you pass an array into a Set, any duplicate values are automatically removed.

The process involves converting the array to a Set and then back to an array. This is a clean and simple way to ensure that the array contains only unique elements.

Syntax:

To remove duplicates using a Set, you can follow this pattern:

const uniqueArray = [...new Set(array)];

Example:

Here’s an example where we remove duplicates from an array of numbers:

const numbers = [1, 2, 2, 3, 4, 4];
const uniqueNumbers = [...new Set(numbers)];

console.log(uniqueNumbers); // [1, 2, 3, 4]

In this example, the Set automatically removes the duplicate 2s and 4s, and the spread operator (...) is used to convert the Set back into an array.

This method is quick and efficient, especially for removing duplicates when you don’t need to worry about the order or the original data types.

Using `filter()` Method

The filter() method in JavaScript creates a new array that includes only the elements that pass a specific test. You can leverage this method to remove duplicates by checking whether the current element is the first occurrence in the array.

To achieve this, you can use indexOf() inside filter(). The indexOf() method returns the first index at which a given element is found in the array. By comparing the current index with the result of indexOf(), you can determine if the element is being encountered for the first time.

Syntax:

Here’s the pattern to use filter() and indexOf() together:

const uniqueArray = array.filter((value, index, self) => self.indexOf(value) === index);

Example:

In this example, we remove duplicates from an array of numbers:

const numbers = [1, 2, 2, 3, 4, 4];
const uniqueNumbers = numbers.filter((value, index, self) => self.indexOf(value) === index);

console.log(uniqueNumbers); // [1, 2, 3, 4]

Here’s how it works:

filter() goes through each element in the numbers array.
indexOf(value) checks where the element first appears in the array.
If the current index is the same as the first occurrence (i.e., indexOf(value)), it means the element is unique so far, and it’s included in the new array.

This method is easy to understand and works well for smaller arrays or cases where maintaining order is important. However, it might be slower for larger arrays since indexOf() is called for every element.

Using `reduce()` Method

The reduce() method in JavaScript allows you to accumulate values in an array based on a function you provide. It can also be used to remove duplicates by checking if an element is already present in the accumulator (the array being built).

With reduce(), we iterate over each element in the array and add it to the accumulator only if it hasn’t been added already. This helps to keep only the unique values in the resulting array.

Syntax:

The basic syntax for using reduce() to remove duplicates looks like this:

const uniqueArray = array.reduce((acc, curr) => {

  if (!acc.includes(curr)) {
    acc.push(curr);
  }

  return acc;

}, []);

Example:

In this example, we remove duplicates from an array of numbers:

const numbers = [1, 2, 2, 3, 4, 4];

const uniqueNumbers = numbers.reduce((acc, curr) => {

  if (!acc.includes(curr)) {
    acc.push(curr);
  }

  return acc;

}, []);

console.log(uniqueNumbers); // [1, 2, 3, 4]

Here’s how it works:

reduce() iterates through each element in the array.
The accumulator (acc) starts as an empty array ([]).
For each element (curr), we check if it’s already in the accumulator using includes().
If it’s not already in the accumulator, we add it with push().
The result is an array of unique values, preserving the order of their first appearance.

This method gives you more control and can be useful in situations where you need to customize how duplicates are handled or if you need to perform additional operations while reducing. However, like the filter() method, it uses includes(), which may be inefficient for larger arrays.

Using `forEach()` Method

The forEach() method allows you to loop through an array and perform an action for each element. It doesn’t return a new array, but it can be used to build one manually. To remove duplicates, you can check if an element is already present in a new array before adding it.

In this case, we will use forEach() to iterate over the original array and only add the element to a new array if it hasn’t already been added.

Syntax:

The basic syntax for using forEach() to remove duplicates looks like this:

const uniqueArray = [];

array.forEach(item => {

  if (!uniqueArray.includes(item)) {
    uniqueArray.push(item);
  }

});

Example:

Here’s how to remove duplicates from an array of numbers using forEach():

const numbers = [1, 2, 2, 3, 4, 4];
const uniqueNumbers = [];

numbers.forEach(num => {

  if (!uniqueNumbers.includes(num)) {
    uniqueNumbers.push(num);
  }

});

console.log(uniqueNumbers); // [1, 2, 3, 4]

In this example:

We loop through each number in the numbers array.
For each number, we check if it’s already present in the uniqueNumbers array using includes().
If it’s not already in the uniqueNumbers array, we add it with push().
The result is an array of unique numbers without any duplicates.

While forEach() is simple and easy to use, keep in mind that it has to check each element using includes(), which can be less efficient for larger arrays. Nevertheless, it’s a good and readable approach for removing duplicates when working with small to moderate-sized arrays.

Using `map()` and `filter()` Combination

You can also combine the map() and filter() methods to remove duplicates from an array. While map() is typically used for transforming arrays, you can use it to create a new array where each element is either the original value or null (if it’s a duplicate). Then, filter() can be used to remove the null values, leaving only the unique elements.

map(): We use map() to iterate over the array and create a new array, checking if the current element’s index is the same as its first occurrence (using indexOf()).
filter(): After applying map(), we use filter() to remove the null values, keeping only the unique elements.

Syntax:

Here’s how you can combine map() and filter() to remove duplicates:

const uniqueArray = array
  .map((value, index, self) => self.indexOf(value) === index ? value : null)
  .filter(Boolean);

Example:

Here’s how to use map() and filter() to remove duplicates from an array of numbers:

const numbers = [1, 2, 2, 3, 4, 4];

const uniqueNumbers = numbers
  .map((value, index, self) => self.indexOf(value) === index ? value : null)
  .filter(Boolean);

console.log(uniqueNumbers); // [1, 2, 3, 4]

In this example:

The map() method iterates over the numbers array, and for each element, it checks if the first occurrence index (indexOf(value)) matches the current index. If it does, the element is retained; otherwise, it’s replaced with null.
Then, the filter(Boolean) removes all null values, leaving only the unique values in the final result.

This method is a good option if you prefer chaining array methods and need a concise, readable approach to removing duplicates.

Using `Array.prototype.indexOf()` with a Loop

You can use the indexOf() method along with a for loop to remove duplicates from an array. The idea is to iterate through the original array and, for each element, check if it already exists in the new array using indexOf(). If it does not exist (i.e., indexOf() returns -1), you can add it to the new array.

indexOf(): This method checks if an element already exists in the array. It returns the index of the element if found, or -1 if the element does not exist.
Looping: The for loop iterates over the original array, and for each element, indexOf() is used on the new array to check if that element is already included.

Syntax:

Here’s how you can use indexOf() with a loop to remove duplicates:

const uniqueArray = [];

for (let i = 0; i < array.length; i++) {

  if (uniqueArray.indexOf(array[i]) === -1) {
    uniqueArray.push(array[i]);
  }

}

Example:

Here’s an example of removing duplicates from an array of numbers using indexOf() and a for loop:

const numbers = [1, 2, 2, 3, 4, 4];
const uniqueNumbers = [];

for (let i = 0; i < numbers.length; i++) {

  if (uniqueNumbers.indexOf(numbers[i]) === -1) {
    uniqueNumbers.push(numbers[i]);
  }

}

console.log(uniqueNumbers); // [1, 2, 3, 4]

In this example:

The loop goes through each element of the numbers array.
The indexOf() method checks if the current element already exists in the uniqueNumbers array.
If it doesn’t exist (i.e., indexOf() returns -1), the element is added to the uniqueNumbers array.

This method is simple and easy to understand, especially when you’re working with small arrays. However, for large arrays, more efficient methods like Set or filter() might be preferred.

Conclusion

In this article, we explored various methods to remove duplicates from arrays in JavaScript. We covered several techniques such as:

Using Set: A simple and effective way to automatically remove duplicates by converting the array into a Set.
Using filter(): A flexible approach that leverages indexOf() to filter out duplicate values.
Using reduce(): A method that accumulates unique values into a new array.
Using forEach(): A straightforward looping method to check and add unique elements.
Using map() and filter(): A combination of both methods to remove duplicates in a single step.
Using indexOf() in a for loop: A manual method to check for duplicates before adding values to the new array.

Each method offers a unique advantage depending on the use case, whether it’s simplicity, flexibility, or manual control over the process.

Remember, JavaScript provides many ways to manipulate arrays, and knowing how to remove duplicates is just one example of the power and flexibility you have when working with arrays. Choose the method that best suits your needs, and experiment with these options to master array manipulation in JavaScript!

Using Set to Remove Duplicates

Using filter() Method

Using reduce() Method

Using forEach() Method

Using map() and filter() Combination

Using Array.prototype.indexOf() with a Loop

Conclusion

Related Posts

Using `Set` to Remove Duplicates

Using `filter()` Method

Using `reduce()` Method

Using `forEach()` Method

Using `map()` and `filter()` Combination

Using `Array.prototype.indexOf()` with a Loop