Java recipe: Find duplicates in your List

Examples to count and receive duplicate objects in a list. Given a list of `Friend` objects with auto generated `Id` and with `equals()` by `name`: ```java private List<Friend> friends; @Before public void setUp() throws Exception { friends = Arrays.asList( new Friend("Joey"), new Friend("Joey"), new Friend("Chandler"), new Friend("Monica"), new Friend("Phoebe"), new Friend("Phoebe"), new Friend("Phoebe"), new Friend("Rachel"), new Friend("Ross") ); } ``` Needed to find and count friends with the same name. <a name='more'></a> ---- ### Count Frequency To count duplicates the most appropriate to use `Collections.freguency`method: ```java @Test public void testDuplicatesCount() throws Exception { Set<Friend> uniqueSet = new HashSet<>(friends); for (Friend friend : uniqueSet) { System.out.println(friend + ": " + Collections.frequency(friends, friend)); } assertTrue(uniqueSet.size() < friends.size()); } ``` Output: ``` Phoebe: 3 Rachel: 1 Joey: 2 Ross: 1 Chandler: 1 Monica: 1 ``` ---- ### Get Duplicates List It's often useful to get duplicates list to work with: ```java @Test public void testDuplicatesList() throws Exception { Map<Friend, Integer> map = new HashMap<>(); List<Friend> duplicates = new ArrayList<>(); for (Friend friend : friends) { Integer count = map.get(friend); // not a first occurrence if (count != null) { duplicates.add(friend); } map.put(friend, (count == null) ? 1 : count + 1); } System.out.println("friends frequency: " + map.toString() + " duplicates: " + duplicates.size()); assertEquals(3, duplicates.size()); } ``` Output: ``` friends frequency: {Phoebe=3, Rachel=1, Joey=2, Ross=1, Chandler=1, Monica=1} duplicates: 3 ``` ---- ### Get Duplicates Map Spread duplicates by name among the separate lists: ```java @Test public void duplicatesList() throws Exception { Map<String, List<Friend>> duplicatesMap = new HashMap<>(); for (Friend friend : friends) { final int frequency = Collections.frequency(friends, friend); if (frequency > 1) { // first occurrence if (!duplicatesMap.containsKey(friend.getName())) { duplicatesMap.put(friend.getName(), new ArrayList<>()); continue; } duplicatesMap.get(friend.getName()).add(friend); } } for (String name : duplicatesMap.keySet()) { System.out.println("duplicates of " + name + " : " + duplicatesMap.get(name).size()); } assertEquals(1, duplicatesMap.get("Joey").size()); assertEquals(2, duplicatesMap.get("Phoebe").size()); } ``` Output: ``` duplicates of Phoebe : 2 duplicates of Joey : 1 ``` ---- ### Find Duplicates right in DB To group by name and display count of names which occur more then once ```sql SELECT name, COUNT(name) FROM friends GROUP BY name HAVING ( COUNT(name) > 1 ); ``` Output: ``` name | COUNT(name) -------------------- Phoebe | 3 Joey | 2 ``` Use [Hamcrest](http://hamcrest.org/JavaHamcrest/) for more advanced Collections evaluation. ---- ## see Also * [Java Recipe: Read resource file as a String](http://bloomlab.blogspot.com.ee/2016/04/java-recipe-read-resource-file-as-string.html) * [Java: Convert JSON to DTO Object](http://bloomlab.blogspot.com.ee/2016/02/java-convert-json-to-dto-object.html) * [Java Collections - API](https://docs.oracle.com/javase/8/docs/api/java/util/Collection.html) * [Hamcrest Matchers](http://hamcrest.org/JavaHamcrest/)