Radix Sort

https://github.com/explorer436/programming-playground/tree/main/java-playground/my-implementations/searching-and-sorting/src/main/java/com/my/company/sorting/algorithms

The lower bound for Comparison based sorting algorithm (Merge Sort, Heap Sort, Quick-Sort .. etc) is Ω(nLogn), i.e., they cannot do better than nLogn.

Counting sort is a linear time sorting algorithm that sort in O(n+k) time when elements are in range from 1 to k.

What if the elements are in range from 1 to n^2? We can’t use counting sort because counting sort will take O(n^2) which is worse than comparison based sorting algorithms. Can we sort such an array in linear time?

Radix Sort is the answer. The idea of Radix Sort is to do digit by digit sort starting from least significant digit to most significant digit. Radix sort uses counting sort as a subroutine to sort.

The Radix Sort Algorithm

Do following for each digit i where i varies from least significant digit to the most significant digit.
1. Sort input array using counting sort (or any stable sort) according to the i’th digit.

What is the running time of Radix Sort?

Let there be d digits in input integers. Radix Sort takes O(d*(n+b)) time where b is the base for representing numbers, for example, for decimal system, b is 10. What is the value of d? If k is the maximum possible value, then d would be O(logb(k)). So overall time complexity is O((n+b) * logb(k)). Which looks more than the time complexity of comparison based sorting algorithms for a large k. Let us first limit k. Let k <= nc where c is a constant. In that case, the complexity becomes O(nLogb(n)). But it still doesn’t beat comparison based sorting algorithms. What if we make value of b larger?. What should be the value of b to make the time complexity linear? If we set b as n, we get the time complexity as O(n). In other words, we can sort an array of integers with range from 1 to nc if the numbers are represented in base n (or every digit takes log2(n) bits).

Is Radix Sort preferable to Comparison based sorting algorithms like Quick-Sort?

If we have log2n bits for every digit, the running time of Radix appears to be better than Quick Sort for a wide range of input numbers. The constant factors hidden in asymptotic notation are higher for Radix Sort and Quick-Sort uses hardware caches more effectively. Also, Radix sort uses counting sort as a subroutine and counting sort takes extra space to sort numbers.

Radix Sort Explanation

Radix Sort is a non-comparison-based sorting algorithm that sorts integers by processing their digits individually, from the least significant digit (LSD) to the most significant digit (MSD) or vice versa. It uses a stable sorting algorithm like Counting Sort as a subroutine to sort elements based on each digit. Radix Sort is efficient for sorting large lists of integers or strings when the number of digits is relatively small.

Pseudocode for Radix Sort (LSD)

RadixSort(arr, n):
    // Input: array arr of size n, non-negative integers
    // Find the maximum number to determine the number of digits
    max_num = maximum value in arr
    d = number of digits in max_num

    // Process each digit from least significant to most significant
    for digit_position from 1 to d:
        // Use a stable sorting algorithm (e.g., Counting Sort) to sort by current digit
        CountingSortByDigit(arr, n, digit_position)

    return arr

CountingSortByDigit(arr, n, digit_position):
    // Input: array arr, digit_position (1 for units, 2 for tens, etc.)
    // Range of digit values is [0, 9], so k = 9
    count = array of size 10 initialized to 0
    output = array of size n

    // Step 1: Extract and count digits
    for i from 0 to n-1:
        digit = (arr[i] / 10^(digit_position-1)) % 10
        count[digit] = count[digit] + 1

    // Step 2: Compute cumulative counts
    for i from 1 to 9:
        count[i] = count[i] + count[i-1]

    // Step 3: Build output array (stable)
    for i from n-1 down to 0:
        digit = (arr[i] / 10^(digit_position-1)) % 10
        output[count[digit] - 1] = arr[i]
        count[digit] = count[digit] - 1

    // Step 4: Copy output back to arr
    for i from 0 to n-1:
        arr[i] = output[i]

Example

Consider the array: [170, 45, 75, 90, 802, 24, 2, 66]

The maximum number is 802, which has 3 digits, so we process 3 digit positions (units, tens, hundreds).

Step-by-Step Process

Sort by Units Digit (digit_position = 1):
- Extract units digit: [0, 5, 5, 0, 2, 4, 2, 6]
- Use Counting Sort:
  - Count: [2, 0, 2, 0, 1, 2, 1, 0, 0, 0] (2 zeros, 2 twos, 1 four, 2 fives, 1 six)
  - Cumulative count: [2, 2, 4, 4, 5, 7, 8, 8, 8, 8]
  - Place elements in output (right to left for stability):
    - 66 (digit 6) → output[7]
    - 2 (digit 2) → output[3]
    - 24 (digit 4) → output[4]
    - 802 (digit 2) → output[2]
    - 90 (digit 0) → output[1]
    - 75 (digit 5) → output[6]
    - 45 (digit 5) → output[5]
    - 170 (digit 0) → output[0]
  - Output: [170, 90, 802, 2, 24, 45, 75, 66]
- Update arr: [170, 90, 802, 2, 24, 45, 75, 66]
Sort by Tens Digit (digit_position = 2):
- Extract tens digit: [7, 9, 0, 0, 2, 4, 7, 6] (note: 2 has no tens digit, so it’s 0)
- Use Counting Sort:
  - Count: [2, 0, 1, 0, 1, 0, 1, 2, 0, 1]
  - Cumulative count: [2, 2, 3, 3, 4, 4, 5, 7, 7, 8]
  - Place elements in output:
    - 66 (digit 6) → output[4]
    - 75 (digit 7) → output[6]
    - 45 (digit 4) → output[3]
    - 24 (digit 2) → output[2]
    - 2 (digit 0) → output[1]
    - 802 (digit 0) → output[0]
    - 90 (digit 9) → output[7]
    - 170 (digit 7) → output[5]
  - Output: [802, 2, 24, 45, 66, 170, 75, 90]
- Update arr: [802, 2, 24, 45, 66, 170, 75, 90]
Sort by Hundreds Digit (digit_position = 3):
- Extract hundreds digit: [8, 0, 0, 0, 0, 1, 0, 0] (numbers < 100 have 0)
- Use Counting Sort:
  - Count: [6, 1, 0, 0, 0, 0, 0, 0, 1, 0]
  - Cumulative count: [6, 7, 7, 7, 7, 7, 7, 7, 8, 8]
  - Place elements in output:
    - 90 (digit 0) → output[5]
    - 75 (digit 0) → output[4]
    - 170 (digit 1) → output[6]
    - 66 (digit 0) → output[3]
    - 45 (digit 0) → output[2]
    - 24 (digit 0) → output[1]
    - 2 (digit 0) → output[0]
    - 802 (digit 8) → output[7]
  - Output: [2, 24, 45, 66, 75, 90, 170, 802]
- Update arr: [2, 24, 45, 66, 75, 90, 170, 802]
Sorted Array: [2, 24, 45, 66, 75, 90, 170, 802]

Complexity analysis

Time Complexity:
- O(d * (n + k)), where:
  - n is the number of elements,
  - d is the number of digits in the maximum number,
  - k is the range of digit values (10 for decimal digits).
- Efficient when d is small relative to n.
Space Complexity: O(n + k) for the output and count arrays in Counting Sort.
Stability: Stable if the subroutine (e.g., Counting Sort) is stable.
Advantages: Efficient for large datasets with a small number of digits, works well for integers and fixed-length strings.
Disadvantages: Requires extra space, only works for integers or mappable values, less efficient if the maximum number has many digits.

Notes

Radix Sort can use either LSD (least significant digit) or MSD (most significant digit) approaches. LSD is more common and simpler to implement.
For negative numbers, preprocess by separating positives and negatives or shifting the range.
Radix Sort is often used in applications like sorting large sets of integers or strings (e.g., in suffix arrays).
The choice of base (e.g., decimal, binary) affects performance; decimal is shown here, but binary (radix 2) or other bases can be used for optimization.

Links to this note

Sorting algorithms