In my work in fraud prevention, preparing a review involves opening several browser tabs across different tools and platforms. We look for the relevant signals and data in those dashboards, write notes for internal reference, and use that information to make a decision.

The gathering and note-taking are necessary, but they also involve repeated manual steps. I wanted to explore how much of that preparation we could automate, so analysts could spend more of their attention on understanding the case.

I proposed and planned an AI-assisted review helper as a team project. We built it as a skill: a reusable workflow that tells an AI agent what information to gather and how to prepare a review note. Its role is limited to gathering and organizing information. It does not make a decision on behalf of a fraud analyst.

I steered the project, and we divided the work among teammates. We discussed which fields would be useful to an analyst and worked out the note format together. One teammate consolidated those discussions into the final format. I wrote the SQL queries to gather the agreed information from our internal data warehouse, while another teammate started building the skill. I then helped complete the skill, tested it, and iterated until we had a working version.

Keeping a human in the loop starts during development. People need to define what information is useful, verify what the queries retrieve, and check the generated notes. Human judgment is required when testing the skill, interpreting feedback, and deciding whether a change improves the output.

One of the hardest parts was deciding how much information the note should contain.

Too much detail becomes a wall of text that the analyst has to work through. Too little makes the note less useful and sends them back to the dashboards to fill in the gaps. We spent a lot of time thinking about that balance and getting the agent to produce the right amount of information consistently.

During testing, I shortened the explanations by removing unnecessary words, a recurring issue with AI-generated text. I also reorganized the signals and data into their relevant sections so related information was easier to find and read together.

I kept the supporting details at the end of the note and indicated that they were there for AI analysis. The purpose is to support future evaluation of two things: whether the collected data is accurate, and whether the note generated from that data is correct and relevant. Any AI-assisted evaluation would also need human verification. Keeping those details available gives us a way to check the output without putting everything into the main review summary.

The facts still come from the underlying data sources. The queries retrieve the information, and the AI-assisted workflow prepares it in the agreed format. Missing or unavailable information also needs to remain visible, so the analyst knows what still needs checking.

Human involvement is equally necessary when using the note. The fraud analyst must go through the case, verify the evidence, consider the context, and decide what action is appropriate. The final decision belongs to the analyst. The AI’s role ends with preparing the information for that review.

From using the working version, I can see an improvement in preparation and fewer manual steps. We have not measured that improvement yet, so I cannot put a number on the time saved.

There is also an important distinction between reducing hands-on work and reducing the total time before a review is ready. The agent still takes time to gather the data and form the note. If an analyst starts the skill and waits for it, I would not claim that this version already delivers a large reduction in overall review time.

One possibility is to move the preparation into a background job that adds a note to the review case before an analyst opens it. That could reduce the waiting and preparation at the point of review. It is a possible next step, and we would still need to check that the information is current and measure whether it reduces the analyst’s work overall, including verification and corrections. Human review and the analyst’s final decision would remain required.

The quality of the notes also needs more iteration. I have seen the value of teammate feedback with another review-note skill I created for domain abuse reviews, where the notes were assessed as useful for that work. I want to keep applying that approach here: let analysts use the output, collect specific feedback, and check whether the changes make the notes more useful.

My experience in fraud prevention helped me recognize the repeated work and shape what the helper needed to produce. Building it involved project planning, SQL, collaboration, and testing, as well as working with AI. The goal is to make the preparation easier, so analysts have more time and attention for deeper analysis and the judgment each case needs.