Spark Scala Sqrt, Cbrt, and Pow

The sqrt, cbrt, and pow functions compute square roots, cube roots, and arbitrary powers of numeric columns. They return Double regardless of input type and behave like Java's Math.sqrt, Math.cbrt, and Math.pow — including how they handle negative inputs and special values like NaN.

sqrt

sqrt returns the square root of a numeric column. It has two signatures:

def sqrt(e: Column): Column

def sqrt(colName: String): Column

Both return a Double column. The string-name overload is a convenience for sqrt(col("colName")).

val df = Seq(
  0.0,
  1.0,
  4.0,
  9.0,
  16.0,
  2.0,
  100.0,
).toDF("value")

val df2 = df
  .withColumn("sqrt", sqrt(col("value")))

df2.show(false)
// +-----+------------------+
// |value|sqrt              |
// +-----+------------------+
// |0.0  |0.0               |
// |1.0  |1.0               |
// |4.0  |2.0               |
// |9.0  |3.0               |
// |16.0 |4.0               |
// |2.0  |1.4142135623730951|
// |100.0|10.0              |
// +-----+------------------+

Perfect squares return clean integer-valued doubles. Anything else returns the closest double-precision approximation — sqrt(2) is irrational, so the trailing digits are just IEEE 754 precision.

Negative Inputs and Nulls

sqrt of a negative number returns NaN, not an error. A null input produces a null output.

val df = Seq(
  Some(4.0),
  Some(-1.0),
  Some(0.0),
  None,
  Some(2.0),
).toDF("value")

val df2 = df
  .withColumn("sqrt", sqrt(col("value")))

df2.show(false)
// +-----+------------------+
// |value|sqrt              |
// +-----+------------------+
// |4.0  |2.0               |
// |-1.0 |NaN               |
// |0.0  |0.0               |
// |null |null              |
// |2.0  |1.4142135623730951|
// +-----+------------------+

If you need to filter out the NaN values from negative inputs, use isnan or guard the call with a when.

cbrt

cbrt returns the cube root of a numeric column. Like sqrt, it has two signatures:

def cbrt(e: Column): Column

def cbrt(columnName: String): Column

Unlike sqrt, cbrt is defined over all real numbers — the cube root of a negative number is a negative real, not NaN.

val df = Seq(
  0.0,
  1.0,
  8.0,
  27.0,
  -8.0,
  -27.0,
  2.0,
).toDF("value")

val df2 = df
  .withColumn("cbrt", cbrt(col("value")))

df2.show(false)
// +-----+------------------+
// |value|cbrt              |
// +-----+------------------+
// |0.0  |0.0               |
// |1.0  |1.0               |
// |8.0  |2.0               |
// |27.0 |3.0               |
// |-8.0 |-2.0              |
// |-27.0|-3.0              |
// |2.0  |1.2599210498948732|
// +-----+------------------+

cbrt(-8) returns -2.0 rather than NaN — useful when working with quantities that can be negative, like signed deviations or differences, where you want a root-style transform without losing the sign.

pow

pow raises one column to the power of another. It is heavily overloaded so you can mix columns and literal Double values without wrapping every literal in lit:

def pow(l: Column, r: Column): Column

def pow(l: Column, r: Double): Column

def pow(l: Double, r: Column): Column

There are also string-name variants that accept column names in place of Column instances (def pow(leftName: String, rightName: String): Column and friends). Pick whichever overload reads most naturally for the call site.

val df = Seq(
  (2.0, 3.0),
  (5.0, 2.0),
  (10.0, 0.0),
  (4.0, 0.5),
  (2.0, -1.0),
  (3.0, 4.0),
).toDF("base", "exponent")

val df2 = df
  .withColumn("power", pow(col("base"), col("exponent")))

df2.show(false)
// +----+--------+-----+
// |base|exponent|power|
// +----+--------+-----+
// |2.0 |3.0     |8.0  |
// |5.0 |2.0     |25.0 |
// |10.0|0.0     |1.0  |
// |4.0 |0.5     |2.0  |
// |2.0 |-1.0    |0.5  |
// |3.0 |4.0     |81.0 |
// +----+--------+-----+

A few things worth noting: anything raised to 0 is 1.0, pow(4, 0.5) is sqrt(4) = 2.0, and negative exponents give reciprocals — pow(2, -1) is 0.5. The result is always a Double even when both inputs are integer-valued.

Compound Growth With a Literal Exponent

A common use is compound growth: a starting principal multiplied by (1 + rate) ^ years. Mixing a column with literals reads cleanly when you reach for lit only where you need to:

val df = Seq(
  ("loan_a", 1000.0),
  ("loan_b", 2500.0),
  ("loan_c", 500.0),
  ("loan_d", 10000.0),
).toDF("name", "principal")

val rate = 0.05
val years = 10

val df2 = df
  .withColumn("future_value", col("principal") * pow(lit(1 + rate), lit(years)))

df2.show(false)
// +------+---------+------------------+
// |name  |principal|future_value      |
// +------+---------+------------------+
// |loan_a|1000.0   |1628.8946267774422|
// |loan_b|2500.0   |4072.236566943606 |
// |loan_c|500.0    |814.4473133887211 |
// |loan_d|10000.0  |16288.946267774423|
// +------+---------+------------------+

The trailing fractional digits are standard IEEE 754 floating-point representation, not something pow introduced. If you need exact decimal results — currency, for instance — round the output or cast to DecimalType before reporting.

For absolute value, see abs. For rounding the results of these computations to integers or a specific scale, see ceil, floor, and rint and round and bround.

Spark Scala Sqrt, Cbrt, and Pow

sqrt

Negative Inputs and Nulls

cbrt

pow

Compound Growth With a Literal Exponent

Related Functions