Home / Useful Patterns / Unique List

Unique List

Finding the unique elements of a list is a common task, often tied to detecting or removing duplicate values. There are two versions and accompanying methods.

Unique elements by key
Unique elements by value

The choice depends on the presence of a “unique key.” That is, if you can construct a unique key from a set of fields, then use option 1, else you will need to use option 2. Prefer option 1 as option 2 will have much worse runtimes as the input grows.

Unique Elements by Key

The goal here is to detect and remove duplicates based on a key or set of fields. The way we do this is to turn the list into a “map” using struct comprehension and then convert it back to a list.

unique-list-by-key.cue

#elem: {
	key: string
	val: _
}

elems: [...#elem] & [
	{key: "a", val: 1},
	{key: "b", val: 1},
	// { key: "a", val: 2 }, // uncomment this line to see error
	{key: "b", val: 1},
]

set: [string]: #elem
set: {
	for e in elems {
		"\(e.key)": e
	}
}

uniq: [ for s in set {s}]

cue eval unique-list-by-key.cue

Unique Elements by Value

The goal here is to detect and remove duplicates based on the entire value. We use the list package and list comprehension.

unique-list-by-val.cue

import "list"

#elem: {
	key: string
	val: _
}

elems: [...#elem] & [
	{key: "a", val: 1},
	{key: "b", val: 1},
	{key: "a", val: 2},
	{key: "b", val: 1},
]

// we compare the current element and add it if it does not appear in the remainder of the list
// in doing so, we add the last unique occurance to the result
uniq: [ for i, x in elems if !list.Contains(list.Drop(elems, i+1), x) {x}]

cue eval unique-list-by-val.cue

#elem: {
	key: string
	val: _
}
elems: [{
	key: "a"
	val: 1
}, {
	key: "b"
	val: 1
}, {
	key: "a"
	val: 2
}, {
	key: "b"
	val: 1
}]
uniq: [{
	key: "a"
	val: 1
}, {
	key: "a"
	val: 2
}, {
	key: "b"
	val: 1
}]

The Function Pattern

Required List Element

Get CUE support and training from Hofstadter

Published 2024-03-31

Edit this page Create page issue Create project issue