keepColumns (Compute)

When To Use

Use keepColumns to project a wide source table down to just the columns your mapping (or the next compute) needs. It is typically applied after a query or command that returns more fields than necessary, or after computes such as duplicateColumn and tableJoin have widened the table. Every column not listed in columnNumbers is discarded; rows are never removed.

keepColumns filters columns; to filter rows, use keepOnlyMatchingLines^[1] or excludeMatchingLines^[2].

Syntax

sources:
  fanSensors:
    type: commandLine
    commandLine: ${file::ipmi-fru.sh}
    computes:
    - type: keepColumns
      columnNumbers: 1,2,3,7,8,9

Properties

Table Transformation Example

With columnNumbers: 1,3,5:

Serialized, fan1;front;Delta F1;1.02;ok becomes fan1;Delta F1;ok.

Recommended Pattern

• Apply keepColumns as the last compute of the pipeline whenever possible: earlier computes reference columns by index, and dropping columns mid-pipeline renumbers everything after it.
• Add a comment listing the kept column names right next to the compute (as IpmiTool and WindowsIpmiTool do), so the mapping's $1, $2, … references stay understandable.
• Trim tableJoin results with keepColumns to remove the duplicated key columns before mapping.

Common Mistakes

• Expecting keepColumns to reorder columns: columnNumbers: 3,1 yields columns 1 then 3, not 3 then 1. Use duplicateColumn or an awk compute when you really need reordering.
• Forgetting that every subsequent compute and the mapping must use the new column numbering, not the original one.
• Listing an index larger than the row width: the compute logs a warning and leaves the table unchanged, which can silently break the mapping.
• Counting columns from 0: columnNumbers is 1-based.

Community Examples

• Cassandra^[3]
• IpmiTool^[4]
• WinStorageSpaces^[5]

From Cassandra (trimming a chain of tableJoin results before mapping), included directly from the connector source:

- type: keepColumns
  columnNumbers: 1,2,4,6,8