Lingquan | Blog

Array Deduplicate in Javascript

Word count: 469Reading time: 2 min
2019/04/02 Share

Common ways to deduplicate array in Javascript

It’s common to be asked to delete duplicates in arrays in Interviews.
Here is a summary for dedpulication of arrays in Javascript.

Method 1: Set

If we can use ES6, we can just simply use Set object to deduplicate.

Code sample:

1
2
3
4
5
6
7
8
9
var arr = [0,0,,0,'true','true',true,true,false,false, undefined,undefined, null,null, NaN, NaN ,'NaN', 'a', 'a',{},{}];
function dedup(arr) {
return Array.from(new Set(arr));
}
console.log(dedup(arr));

// or:
[...new Set(arr)]
//[0, "true", true, 15, false, undefined, null, NaN, "NaN", "a", {}, {}]

Notes:

  • The Array.from() method creates a new, shallow-copied Array instance from an array-like or iterable object.
  • Disadvantage: Cannot delete duplicate of {};

Method 2: for loop

Use two for loop to check and find the duplicates and then splice the array

Sample code

1
2
3
4
5
6
7
8
9
10
11
12
13
var arr = [0,0,,0,'true','true',true,true,false,false, undefined,undefined, null,null, NaN, NaN ,'NaN', 'a', 'a',{},{}];

function dedup(arr) {
for (var i = 0; i < arr.length; i++) {
for(var j = i + 1; j < arr.length; j++) {
if (arr[i] == arr[j]) {
arr.splice(i, 1);
j--;
}
}
}
return arr;
}

Notes:

  • The splice() method changes the contents of an array by removing or replacing existing elements and/or adding new elements in place.
  • array.splice(start[, deleteCount[, item1[, item2[, …]]]])

Method 3: fliter()

use filter to judge if the current index is the index of the first occureance of the element.

Sample code

1
2
3
4
5
function dedup(arr) {
return arr.filter(function(item, index, arr) {
return arr.indexOf(item, 0) === index;
});
}

Notes:

  • The filter() method creates a new array with all elements that pass the test implemented by the provided function.
  • callback is invoked with three arguments:
    • the value of the element
    • the index of the element
    • the Array object being traversed

##Method 4: sort
Sort the array and compare the neighbors iteratively

Sample Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
function dedup(arr) {
arr = arr.sort()
// must add 1st element
var new_arr= [arr[0]];
for (var i = 1; i < arr.length; i++) {
if (arr[i] !== arr[i-1]) {
new_arr.push(arr[i]);
}
}
return new_arr;
}
var arr = [1,1,'true','true',true,true,15,15,false,false, undefined,undefined, null,null, NaN, NaN,'NaN', 0, 0, 'a', 'a',{},{}];
console.log(unique(arr))

Notes:

  • must add the first element into new array

Method 5: indexof()/includes()

Create a new array and judge if the new array contains the elements

Sample code:

1
2
3
4
5
6
7
8
9
function unique(arr) {
var new_arr = [];
for (var i = 0; i < arr.length; i++) {
if (new_arr .indexOf(arr[i]) === -1) {
new_arr.push(arr[i])
}
}
return new_arr;
}
CATALOG
  1. 1. Common ways to deduplicate array in Javascript
    1. 1.1. Method 1: Set
      1. 1.1.0.1. Code sample:
      2. 1.1.0.2. Notes:
  2. 1.2. Method 2: for loop
  3. 1.3. Sample code
    1. 1.3.0.1. Notes:
  • 1.4. Method 3: fliter()
    1. 1.4.0.1. Sample code
    2. 1.4.0.2. Notes:
    3. 1.4.0.3. Sample Code:
    4. 1.4.0.4. Notes:
  • 1.5. Method 5: indexof()/includes()
    1. 1.5.0.1. Sample code: