How do I add default values to complex elements of an array in a jolt transformation

General
1

I have input that looks like this:

{
  "INS-2000_loop": [
    {
      "HD-2300_loop": [
        {
          "COB-2320_loop": [
            {
              "COB_01": "11",
              "COB_02": "12",
              "COB_03": "13"
            },
            {
              "COB_01": "21",
              "COB_03": "23"
            },
            {
              "COB_01": "31",
              "COB_02": "32"
            },
            {
              "COB_02": "42",
              "COB_03": "43"
            }
          ]
        }
      ]
    }
  ]
}

Currently my spec looks like this:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "HD-2300_loop": {
            "*": {
              "COB-2320_loop": {
                "*": {
                  "COB_01": "L2320.[&1].x",
                  "COB_02": "L2320.[&1].y",
                  "COB_03": "L2320.[&1].z"
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "default",
    "spec": {
      "L2320": [
        { "x": null, "y": null, "z": null }
      ]
    }
  }
]

which gives this result:

{
  "L2320": [
    {
      "x": "11",
      "y": "12",
      "z": "13"
    },
    {
      "x": "21",
      "z": "23"
    },
    {
      "x": "31",
      "y": "32"
    },
    {
      "y": "42",
      "z": "43"
    }
  ]
}

I would like the result to have null defaults like this:

{
  "L2320": [
    {
      "x": "11",
      "y": "12",
      "z": "13"
    },
    {
      "x": "21",
      "y": null,
      "z": "23"
    },
    {
      "x": "31",
      "y": "32",
      "z": null
    },
    {
      "x": null,
      "y": "42",
      "z": "43"
    }
  ]
}

There may be any number of elements in the input array (though likely no more than 5 or 6). Is there a way for me to specify default values for the complex element attributes?

2

To achieve your desired output with default values (null) for missing attributes, you can adjust your Jolt transformation specification to explicitly provide those defaults. Here’s how you can do it:

Revised Jolt Spec

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "HD-2300_loop": {
            "*": {
              "COB-2320_loop": {
                "*": {
                  "COB_01": "L2320.[&1].x",
                  "COB_02": "L2320.[&1].y",
                  "COB_03": "L2320.[&1].z"
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "default",
    "spec": {
      "L2320": {
        "*": {
          "x": null,
          "y": null,
          "z": null
        }
      }
    }
  }
]

Explanation of Changes

  1. First Operation (shift):
  • The structure is transformed to map COB_01, COB_02, and COB_03 into x, y, and z, respectively, in the L2320 array.
  1. Second Operation (default):
  • Default values (null) are assigned to the keys x, y, and z for every element in the L2320 array.
  • The use of "*": { "x": null, "y": null, "z": null } ensures that for every element in the L2320 array, missing keys are filled with null.

Result

With the revised spec, your input:

{
  "INS-2000_loop": [
    {
      "HD-2300_loop": [
        {
          "COB-2320_loop": [
            {
              "COB_01": "11",
              "COB_02": "12",
              "COB_03": "13"
            },
            {
              "COB_01": "21",
              "COB_03": "23"
            },
            {
              "COB_01": "31",
              "COB_02": "32"
            },
            {
              "COB_02": "42",
              "COB_03": "43"
            }
          ]
        }
      ]
    }
  ]
}

Will transform into the desired output:

{
  "L2320": [
    {
      "x": "11",
      "y": "12",
      "z": "13"
    },
    {
      "x": "21",
      "y": null,
      "z": "23"
    },
    {
      "x": "31",
      "y": "32",
      "z": null
    },
    {
      "x": null,
      "y": "42",
      "z": "43"
    }
  ]
}

Key Takeaway

Using the default operation in Jolt allows you to define default values for missing attributes in your output. The combination of shift and default operations ensures that all elements have the desired structure with defaults where necessary.

3

The provided solution demonstrates how to transform JSON input into a structure where missing attributes are filled with default null values using a Jolt specification. Here’s a concise recap of the steps and their purpose:

Jolt Spec Overview

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "HD-2300_loop": {
            "*": {
              "COB-2320_loop": {
                "*": {
                  "COB_01": "L2320.[&1].x",
                  "COB_02": "L2320.[&1].y",
                  "COB_03": "L2320.[&1].z"
                }
              }
            }
          }
        }
      }
    }
  },
  {
    "operation": "default",
    "spec": {
      "L2320": {
        "*": {
          "x": null,
          "y": null,
          "z": null
        }
      }
    }
  }
]

Explanation

1. Shift Operation

  • The shift operation restructures the input JSON.
  • Maps COB_01, COB_02, and COB_03 into the L2320 array as x, y, and z respectively.
  • [&1] ensures the indexed placement of each object in the resulting array (L2320).

2. Default Operation

  • Adds missing keys (x, y, z) to every element of the L2320 array and assigns null as the default value.
  • "*": { "x": null, "y": null, "z": null } ensures all elements of the array follow the same structure.

Input Example

{
  "INS-2000_loop": [
    {
      "HD-2300_loop": [
        {
          "COB-2320_loop": [
            { "COB_01": "11", "COB_02": "12", "COB_03": "13" },
            { "COB_01": "21", "COB_03": "23" },
            { "COB_01": "31", "COB_02": "32" },
            { "COB_02": "42", "COB_03": "43" }
          ]
        }
      ]
    }
  ]
}

Expected Output

{
  "L2320": [
    { "x": "11", "y": "12", "z": "13" },
    { "x": "21", "y": null, "z": "23" },
    { "x": "31", "y": "32", "z": null },
    { "x": null, "y": "42", "z": "43" }
  ]
}

Key Takeaways

  • Combining shift and default: This approach ensures every element in the output has consistent keys with values assigned or set to null.
  • Scalability: This spec can handle varying input sizes with ease, as the default operation applies to all indexed elements.

This methodology ensures that you achieve a complete and uniform output structure.