Practical Higher-Order Function Examples in Data Processing

Higher-order functions are most useful when they disappear into ordinary data work.

You are not trying to write “functional code” for its own sake. You are trying to make data movement easier to read, test, and change.

This page ties together the chapter so far:

• use filter for selection
• use map for transformation
• use reduce for aggregation
• create a custom higher-order function only when it names a repeated pattern

Example 1: Process Log Events

Start with structured log data, not raw strings. That keeps the example close to how JVM services usually handle parsed logs.

1(def log-events
2  [{:level :info  :service "billing" :message "started"}
3   {:level :error :service "billing" :message "card declined"}
4   {:level :warn  :service "api"     :message "slow request"}
5   {:level :error :service "api"     :message "timeout"}])

Now write the steps as small functions:

 1(defn error? [event]
 2  (= :error (:level event)))
 3
 4(defn service-name [event]
 5  (:service event))
 6
 7(defn error-counts-by-service [events]
 8  (->> events
 9       (filter error?)
10       (map service-name)
11       frequencies))

The pipeline is easy to read because each higher-order function has a clear job:

This is the shape you want in production: small rules, clear flow, no hidden mutation.

Example 2: Transform Records For An API Response

Suppose internal order data has more fields than the API should expose.

 1(defn public-order [order]
 2  {:id (:order/id order)
 3   :status (:order/status order)
 4   :total (:order/total order)})
 5
 6(defn public-orders [orders]
 7  (->> orders
 8       (filter :order/visible?)
 9       (map public-order)
10       (into [])))

This keeps the boundary explicit:

• filter enforces visibility
• map shapes the public response
• into [] returns the concrete collection shape expected by the API layer

That is usually clearer than mutating DTOs in a loop.

Example 3: Summarize Transactions

Aggregation is where reduce usually belongs.

1(defn add-transaction [summary {:transaction/keys [kind amount]}]
2  (update summary kind (fnil + 0M) amount))
3
4(defn totals-by-kind [transactions]
5  (reduce add-transaction {} transactions))

This is a direct replacement for a Java loop that updates a map as it walks a list.

The accumulator is explicit, and the update rule is testable on its own.

Example 4: Create A Reusable Processing Shape

If you repeatedly validate records and keep only the valid results, a custom higher-order function can help.

1(defn keep-valid [valid? explain coll]
2  (->> coll
3       (map (fn [item]
4              (if (valid? item)
5                {:status :valid :value item}
6                {:status :invalid :errors (explain item)})))
7       (into [])))

Usage:

1(defn positive-total? [order]
2  (pos? (:order/total order)))
3
4(defn total-errors [order]
5  (when-not (positive-total? order)
6    ["Order total must be positive"]))
7
8(keep-valid positive-total? total-errors orders)

This is worth a custom function because it names a repeated workflow:

• run a validation rule
• return either a valid result or an error result
• preserve the one input to one output shape

Example 5: Combine The Tools Without Collapsing Them

Do not make one giant function do everything.

Prefer a pipeline where each step has a role:

 1(defn reportable? [order]
 2  (and (:order/paid? order)
 3       (:order/visible? order)))
 4
 5(defn report-row [order]
 6  {:id (:order/id order)
 7   :customer (:order/customer-id order)
 8   :total (:order/total order)})
 9
10(defn report-total [rows]
11  (reduce (fn [total row]
12            (+ total (:total row)))
13          0M
14          rows))
15
16(defn build-report [orders]
17  (let [rows (->> orders
18                  (filter reportable?)
19                  (map report-row)
20                  (into []))]
21    {:rows rows
22     :total (report-total rows)}))

This is still small, but it has real structure:

• predicates name selection rules
• mapping functions name output shapes
• reducing functions name aggregation rules
• the final function composes those pieces

Java Migration Notes

The code often becomes shorter, but that is not the main point. The main point is that each function can be reviewed and tested in isolation.

Knowledge Check