Finding duplicates in a data set is a common operation. Whether you're checking for duplicate email addresses in an imported CSV, ensuring user input isn't a repeat, or verifying SKUs in ecommerce app, you've likely needed to find duplicates before.
In Laravel, you traditionally have to chain together a few methods such as countBy() and filter().
Laravel Collections now have a duplicates() method that makes it easy to find all of the duplicate values in a collection, along with their original array indexes.
Table of Contents
The Old Way: countBy() and filter()
Before duplicates() was introduced, we typically groups the data and filters them as shown below.
$emails = collect(['a@test.com', 'b@test.com', 'a@test.com', 'c@test.com']);
// The verbose workaround
$duplicates = $emails->countBy()
->filter(fn ($count) => $count > 1)
->keys();
// Returns: ['a@test.com']
Although it works fine, but it does not tracks the array key which was storing these duplicate record. Which will be difficult in real-world scenarios where during Excel import we need to check which row contains duplicate value.
Basic Usage on Simple Collections
The duplicates() method searches the collection and returns only the values that appear more than once:
$tags = collect(['php', 'laravel', 'mysql', 'php', 'redis', 'laravel']);
$duplicateTags = $tags->duplicates();
// Output:
// [
// 3 => 'php',
// 5 => 'laravel',
// ]
Notice that Laravel keeps the original index at which the duplicate was found. So the index 0 had the first 'php' and index 3 is marked as the duplicate entry. This can be useful if you need to report the exact row number when importing a file.
Finding Duplicates by Key or Property
When working with arrays of associative data or Eloquent collections, pass the column or property name as an argument:
$customers = collect([
['id' => 1, 'email' => 'rahul@example.com'],
['id' => 2, 'email' => 'priya@example.com'],
['id' => 3, 'email' => 'rahul@example.com'], // Duplicate
]);
$duplicateEmails = $customers->duplicates('email');
// Output:
// [
// 2 => 'rahul@example.com'
// ]
You can quickly check if an import has any duplicates before touching the database:
if ($customers->duplicates('email')->isNotEmpty()) {
throw new Exception('The uploaded list contains duplicate emails.');
}
Using a Callback for Custom Matching
If you need case-insensitive matching or want to normalize data before checking for duplicates, pass a closure to duplicates():
$promoCodes = collect(['SUMMER20', 'summer20', 'WINTER50']);
// Case-insensitive duplicate check
$duplicates = $promoCodes->duplicates(function ($code) {
return strtolower(trim($code));
});
// Output:
// [
// 1 => 'summer20'
// ]
Summary
| Usage | Example | Key Behavior |
|---|---|---|
| Simple Values | $collection->duplicates() |
Flags second and subsequent occurrences of scalars. |
| By Attribute | $collection->duplicates('sku') |
Finds duplicates across model attributes or array keys. |
| Custom Callback | $collection->duplicates(fn ($i) => ...) |
Normalizes items before evaluating duplication. |
So basically doing lots of manual works for finding duplicates like grouping, looping, filtering, we can simply use the duplicates() method to find duplicates in a collection easily.
Thank you for reading this article 😊
For any query, do not hesitate to comment 💬