Skip to content
The Chat Archive
deepseek-v3-0324· DeepSeek LMArena Human Preference 140K

it works but input set can have different order. How to make sure that we merge by field…

  1. User

    it works but input set can have different order. How to make sure that we merge by field in order?

  2. Assistant

    If you want to ensure that the order of merging is preserved when deduplicating by fieldA, you need to control which object is kept when duplicates are encountered.

    Problem with Collectors.toMap()

    By default, Collectors.toMap() does not guarantee the order of elements. If you want to preserve insertion order, you should use a LinkedHashMap:

    ✅ Solution 1: Collectors.toMap() with LinkedHashMap (Preserves Insertion Order)

    List<Foo> distinctByFieldA = fooList.stream()
        .collect(Collectors.toMap(
            Foo::getFieldA,      // Key: fieldA
            foo -> foo,          // Value: Foo object
            (existing, replacement) -> existing,  // Keep first occurrence
            LinkedHashMap::new   // Preserves insertion order
        ))
        .values()
        .stream()
        .collect(Collectors.toList());
    
    • LinkedHashMap ensures the order matches the original list.
    • (existing, replacement) -> existing keeps the first occurrence of each fieldA.
      • If you want the last occurrence instead, use (existing, replacement) -> replacement.

    ✅ Solution 2: filter() + Set (Preserves Order & Efficient)

    If you just want to keep the first occurrence and maintain order, this is simpler and faster:

    Set<String> seen = new HashSet<>();
    List<Foo> distinctByFieldA = fooList.stream()
        .filter(foo -> seen.add(foo.getFieldA()))  // Only keeps first occurrence
        .collect(Collectors.toList());
    
    • seen.add(foo.getFieldA()) returns false if fieldA was already in the set, so only the first occurrence is kept.
    • Faster than toMap() for large lists since it avoids intermediate Map creation.

    ✅ Solution 3: Collectors.groupingBy() (If You Need More Control)

    If you need to choose which object to keep (e.g., based on another field), you can use groupingBy with a downstream collector:

    List<Foo> distinctByFieldA = fooList.stream()
        .collect(Collectors.groupingBy(
            Foo::getFieldA,
            LinkedHashMap::new,  // Preserves order
            Collectors.collectingAndThen(
                Collectors.toList(),
                list -> list.get(0)  // Keep first (or pick by some logic)
            )
        ))
        .values()
        .stream()
        .collect(Collectors.toList());
    
    • Useful if you need to apply custom logic (e.g., keep the object with the highest fieldB).

    Summary: Which to Use?

    MethodPreserves OrderPerformanceBest When
    toMap(LinkedHashMap)✅MediumNeed to choose between first/last occurrence
    filter() + Set✅FastestJust need first occurrence
    groupingBy()✅SlowerNeed complex deduplication logic

    Example: Keep Last Occurrence Instead of First

    If you want the last occurrence of each fieldA:

    List<Foo> distinctByFieldA = fooList.stream()
        .collect(Collectors.toMap(
            Foo::getFieldA,
            foo -> foo,
            (existing, replacement) -> replacement,  // Keep last occurrence
            LinkedHashMap::new
        ))
        .values()
        .stream()
        .collect(Collectors.toList());
    

    Would you like to modify the logic further (e.g., keep the object with the highest/lowest value in another field)?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.