Remove Duplicate Words from Sentence Java | 2 Methods

Introduction

Removing duplicate words while preserving order is useful for text cleaning and data normalization.

Method 1: Using LinkedHashSet

import java.util.LinkedHashSet;
import java.util.Set;

public class RemoveDuplicateWords {
    
    public static String removeDuplicateWords(String sentence) {
        if (sentence == null || sentence.isEmpty()) {
            return sentence;
        }
        
        String[] words = sentence.split(" ");
        Set<String> seen = new LinkedHashSet<>();
        
        for (String word : words) {
            seen.add(word.toLowerCase());
        }
        
        return String.join(" ", seen);
    }
    
    public static void main(String[] args) {
        System.out.println(removeDuplicateWords("hello world hello java world"));
        // Output: hello world java
    }
}

Output:

hello world java

Method 2: Using Streams

import java.util.Arrays;
import java.util.stream.Collectors;

public class RemoveDuplicateWordsStream {
    
    public static String removeDuplicateWords(String sentence) {
        return Arrays.stream(sentence.split(" "))
                    .distinct()
                    .collect(Collectors.joining(" "));
    }
}

Frequently Asked Questions

Q1. Should order be preserved?

Answer: Yes, use LinkedHashSet.

Q2. Case sensitivity?

Answer: Convert to lowercase for comparison.

Q3. Punctuation handling?

Answer: Remove or normalize before processing.

Q4. Performance?

Answer: Both O(n), similar speed.

Q5. Keep multiple occurrences?

Answer: Don't use distinct() or set.

Q6. Large text?

Answer: Streams handle memory better.

Q7. Filter specific words?

Answer: Add condition in loop/stream.

Q8. Count duplicates?

Answer: Track removed count separately.

Q9. Consecutive duplicates only?

Answer: Different logic with previous word tracking.

Q10. Multiple spaces between?

Answer: Use regex split with " +".

Conclusion

LinkedHashSet preserves order. Streams more modern. Choose based on Java version and requirements.