Spark Scala Reverse

The reverse function reverses the character order of a string column. It also works on array columns, reversing the element order — but this article focuses on string usage.

The reverse function is defined as:

def reverse(e: Column): Column

It takes a single column and returns a new column with the characters in reversed order. For arrays, it reverses the element order instead.

Reversing Strings

val df = Seq(
  "Spark",
  "Scala",
  "DataFrame",
  "reverse",
).toDF("word")

val df2 = df
  .withColumn("reversed", reverse(col("word")))

df2.show(false)
// +---------+---------+
// |word     |reversed |
// +---------+---------+
// |Spark    |krapS    |
// |Scala    |alacS    |
// |DataFrame|emarFataD|
// |reverse  |esrever  |
// +---------+---------+

Palindrome Detection

Comparing a string to its reversed form is a straightforward way to check for palindromes:

val df = Seq(
  "madam",
  "racecar",
  "hello",
  "level",
  "spark",
).toDF("word")

val df2 = df
  .withColumn("reversed", reverse(col("word")))
  .withColumn("is_palindrome", col("word") === col("reversed"))

df2.show(false)
// +-------+--------+-------------+
// |word   |reversed|is_palindrome|
// +-------+--------+-------------+
// |madam  |madam   |true         |
// |racecar|racecar |true         |
// |hello  |olleh   |false        |
// |level  |level   |true         |
// |spark  |kraps   |false        |
// +-------+--------+-------------+

For case-insensitive palindrome checks, apply lower to both sides before comparing.

Handling Nulls and Empty Strings

When reverse encounters a null, the result is null. An empty string reverses to an empty string:

val df = Seq(
  ("Alice", "hello"),
  ("Bob", null),
  (null, "world"),
  ("Dave", ""),
).toDF("name", "greeting")

val df2 = df
  .withColumn("name_reversed", reverse(col("name")))
  .withColumn("greeting_reversed", reverse(col("greeting")))

df2.show(false)
// +-----+--------+-------------+-----------------+
// |name |greeting|name_reversed|greeting_reversed|
// +-----+--------+-------------+-----------------+
// |Alice|hello   |ecilA        |olleh            |
// |Bob  |null    |boB          |null             |
// |null |world   |null         |dlrow            |
// |Dave |        |evaD         |                 |
// +-----+--------+-------------+-----------------+

For other string transformation functions, see lower and upper for case conversion, translate for character-by-character substitution, or replace for substring replacement.