To find (and not delete) duplicates in a JavaScript array, you can use this function:
function showDupes(arr) {
return [...new Set(arr.filter((elem, idx, arr) => arr.indexOf(elem) !== idx))]
}
Example use:
const dupes = showDupes([1,1,1,2,2,2,3,4,5,6]) console.log(dupes)
Output:
[ 1, 2 ]
To learn how the above approach works, and what alternatives exist, please keep on reading. This guide shows you ways to find array duplicates and show them instead of deleting them. Besides, you will learn what performance issues there are with the naive approach and how to write a much quicker approach.
Let’s jump into it.
How to Find Duplicates in JavaScript Arrays
Given an array [1, 2, 4, 2, 5, 5, 5] you can instantly see that the values 2 and 5 are duplicates. But to let the JavaScript program know this, it must go through the values one by one and compare those with the rest of the values.
To see the values that are duplicated in a JavaScript array:
- Pick the first array element and compare it with the rest of the elements one by one.
- Pick the second element and compare it with the rest of the elements again.
- This process repeats as long as there are values in the array.
To do this, you need a loop that goes through the elements one by one. Then inside that for loop, you need another loop that goes through the rest of the values one by one and compares them with the current number.
The For-Loop Approach
Here’s a nested for loop that implements the logic described above.
const array = [1, 2, 4, 2, 5, 5, 5];
const duplicates = [];
for (let i = 0; i < array.length; i++) {
for (let j = i + 1; j < array.length; j++) {
if (array[i] === array[j]) {
duplicates.push(array[i]);
}
}
}
console.log(duplicates)
Output:
[ 2, 5, 5, 5 ]
This for-loop iterates through each element in the array and compares it to every other element in the array. If it finds a match, it adds the element there and jumps to check the next element. This process continues until there are no elements left in the array.
Now as you can instantly see, this approach works but it shows the duplicate values multiple times. To understand why this happens, just look at the image you saw earlier. For example, the 1st number 5 compares itself with the 2nd and 3rd number 5s and the 2nd number 5 compares itself with the 3rd number 5. This is why there are three 5s in the result.
To only show the duplicated value once, you need to keep track of the duplicate values already seen and make sure not to add an existing duplicate to the duplicates array.
The Improved For-Loop Approach
Here’s the improved version of the above for loop:
const array = [1, 2, 2, 4, 3, 4, 4, 4, 4, 4, 4, 5, 5, 5];
const duplicates = [];
for (let i = 0; i < array.length; i++) {
for (let j = i + 1; j < array.length; j++) {
if (array[i] === array[j] && !duplicates.includes(array[i])) {
duplicates.push(array[i]);
}
}
}
console.log(duplicates)
Output:
[ 2, 4, 5 ]
It works the exact same way as the previous for loop but it also checks that the duplicates array doesn’t already have the duplicate about to be added there (!duplicates.includes(array[i])).
The for-loop approach is a straightforward way to check what values occur in the array more than once. But if you’re looking for an alternative approach with less code, you can also implement the nested for-loop structure with the built-in array filter() function.
The Filter Method Approach
Here’s how you can check duplicate values with array filter() method:
const array = [1, 2, 4, 2, 5, 5, 5]; const duplicates = array.filter((element, index, arr) => arr.indexOf(element) !== index) console.log(duplicates)
Output:
[ 2, 5, 5 ]
Here the filter() method takes each element of the array to an inspection one by one. In a sense, it’s the outer for loop when compared to the previous solution. The filter() method runs the function passed as an argument for each element to see if the element exists somewhere else in the array.
Thus, the indexOf() method is the inner for-loop. It checks the first index at which the current element occurs in the array.
Improved Filter Approach
Similar to the initial for-loop approach, this approach also shows duplicate values multiple times.
To fix the problem, you can place the duplicates into a set and convert the set back to an array. This works because a JavaScript set will automatically remove duplicate values.
const array = [1, 2, 4, 2, 5, 5, 5]; const duplicates = array.filter((element, index, arr) => arr.indexOf(element) !== index) const unique_duplicates = [... new Set(duplicates)] console.log(unique_duplicates)
Output:
[ 2, 5 ]
Create a Function
Last but not least, to make things look professional, it makes sense to place the filtering functionality into a separate JavaScript function:
function showDupes(arr) {
return [...new Set(arr.filter((elem, idx, arr) => arr.indexOf(elem) !== idx))]
}
This is the piece of code you already saw in the introduction.
Performance
Both the for-loop and the filter approach are rather naive and perform poorly. This is because the array data is unsorted and a lot of unnecessary checks need to take place before you can know the duplicates.
For example, let’s measure how long it takes to check an array with 100,000 elements for duplicates:
const arr = [...Array(100_000).keys()].fill(0, 2, 4).fill(9, 10, 14)
console.time("time")
console.log(arr.filter((e, i, a) => a.indexOf(e) !== i))
console.timeEnd("time")
Output:
time: 4080.6533203125 ms
On my setup, it takes 4+ seconds.
This is a really poor performance and gets worse as the array size grows. In case you’re looking for an efficient way to check what are the duplicates in the array, you need a faster approach.
Solution: Sort First
One way to avoid doing excess comparisons is by sorting the data first and then using the filter on it.
Also, keep in mind the data is always not sortable. With the basic example of numbers, we can quite easily sort the data in an increasing/decreasing order.
When sorted, you can just compare a value to the previous neighbor. If the neighboring value is the same, it’s a duplicate. This way the filter() method only needs to run the function through once and check each consecutive numbers instead of comparing all numbers with the rest of the numbers.
Assuming the sorting algorithm was quick, this is a huge time-saver.
Here’s the better-performing approach that sorts the array first and then compares the neighbors only:
const arr = [...Array(100_000).keys()].fill(0, 2, 4).fill(9, 10, 14)
console.time("time")
console.log(arr.sort((a, b) => a > b ? 1 : -1).filter((e, i, a) => a[i-1] === e))
console.timeEnd("time")
Output:
time: 2.742919921875 ms
On my setup, this only took ~3ms which is 0.003 seconds, which is 1000+ times faster than the naive filter approach without sort.
Thanks for reading. Happy coding!