Jolt Spec to filter multiple value in json field

General
1

I have the following json and would like to filter few object only -

  {
    "field1": "xyz",
    "field2": "mno",
    "res1": "pqrs",
    "folder": "/folder-parent/raw/new"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/raw/old"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/changed-per-record"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/something-else"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/something/other-thing"
  }
]

I need filtered objects with /folder-parent/raw/new /folder-parent/raw/old /folder-parent/change-per-record

I tried this but does not work [ { "operation": "shift", "spec": { "*": { "folder": { "*": { "filter": "@(2, folder)=== '/folder-parent/raw/new' && @(2, folder)=== '/folder-parent/raw/old' && @(2, folder)=== '/folder-parent/change-per-record'", "": "@" } } } } } ] Please help in this case

2

To filter the objects in the JSON based on specific values of the folder field, you can use JavaScript, Jolt, or other tools. However, the Jolt transformation you shared won’t work because it doesn’t support direct conditional logic like combining multiple conditions with &&.

Here’s how you can solve this problem in two ways:

1. Using JavaScript

You can use the filter() method in JavaScript to filter the objects from the JSON array:

const data = [
  {
    "field1": "xyz",
    "field2": "mno",
    "res1": "pqrs",
    "folder": "/folder-parent/raw/new"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/raw/old"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/changed-per-record"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/something-else"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/something/other-thing"
  }
];

const filtered = data.filter(item =>
  ["/folder-parent/raw/new", "/folder-parent/raw/old", "/folder-parent/changed-per-record"].includes(item.folder)
);

console.log(filtered);

Output:

[
  {
    "field1": "xyz",
    "field2": "mno",
    "res1": "pqrs",
    "folder": "/folder-parent/raw/new"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/raw/old"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/changed-per-record"
  }
]

2. Using Jolt Transformation

If you want to filter using Jolt, you need to use the shift transformation and explicitly check for each matching folder value.

Here’s the correct Jolt specification:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "folder": {
          "/folder-parent/raw/new|/folder-parent/raw/old|/folder-parent/changed-per-record": {
            "@2": "[]"
          }
        }
      }
    }
  }
]

Explanation of the Spec:

  1. The wildcard "*" matches all objects in the array.
  2. The folder key is checked against a regex-like condition:
    "/folder-parent/raw/new|/folder-parent/raw/old|/folder-parent/changed-per-record".
  3. When a match is found, the entire object (@2) is added to the output array ([]).

Output (Filtered JSON):

[
  {
    "field1": "xyz",
    "field2": "mno",
    "res1": "pqrs",
    "folder": "/folder-parent/raw/new"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/raw/old"
  },
  {
    "field1": "xyz1",
    "field2": "mno1",
    "res1": "pqrs1",
    "folder": "/folder-parent/changed-per-record"
  }
]

Choose the Right Approach

  • JavaScript is simpler and more flexible, especially if you’re working in a JavaScript-based environment.
  • Jolt is useful if you’re working with Jolt-compatible systems (e.g., Apache Nifi or JSON transformation pipelines).

Let me know if you need further clarification!