Process Data
Purpose: Process Data is a Skill for working with larger files. If you already know Transform Data, you pretty much already know how to use it. This article covers what it does, the one difference to know about, and how to decide between Process Data and Transform Data.
What it is
You'll find Process Data as a tile in the Skill Library, and you add it to your Agent the same way you'd add any other Skill. It's built for larger files, roughly 50MB and up, that would otherwise run into memory or runtime limits in Transform Data.
The configuration panel is nearly identical to Transform Data, so there's almost nothing new to learn to get going.
The Process Data tile in the Skill Library.
The one thing that's different
Process Data keeps your values, not your formatting.
Process Data strips Excel formatting and formulas, so only the values come through. You'll see a permanent banner in the panel as a reminder, so there are no surprises when you look at the output.
The permanent banner inside the Process Data panel.
When to reach for each
Process Data LARGE FILES
Use it to crunch a large file down to a manageable size, fast. This is your heavy-lifting step.
Transform Data REVIEWABLE OUTPUT
Use it when you need to compile your data into a clean, reviewable Excel spreadsheet.
Plenty of workflows use both. Let Process Data handle the heavy lifting on a big file, then hand off to Transform Data to shape what's left into something a reviewer can open and read.
Keep in mind
Bigger, but not unlimited
Process Data doesn't remove every limit. Each step still has its own runtime cap, so a very large or complex job can still hit it.
Design the whole workflow
Mixing Process and Transform doesn't guarantee the workflow runs. Keep Process focused on shrinking large files, and use Transform only when you need reviewable output.
Still sample your files
When you hand a file to the Builder, give it a sample of your data rather than the whole thing.
Beta
Process Data is in Beta. If you have questions, reach out to your FDA.