Meet Scott from Devonport City Council in Australia, who just walked us through his incredible journey of creating Mersey, a Travel AI Agent.

Meet the builder behind the lighthouse

Scott is an enterprise solutions architect who’s worked at Devonport City Council for almost seven years, with dual degrees in criminology, criminal justice, and business management.

He built Mersey to help users landing on the website, and this blog is about his learnings and journey.

Version 1: The struggle was real

Scott started with what seemed like a simple setup. Mersey version 1 used Tars’ web knowledge base tool. The flow was straightforward: the user greets Mersey, asks a question, and Mersey pulls information from the knowledge base. Simple, right?

“The web crawler had to be manually activated and it was a bit I found it a bit tiresome to keep activating when we added new content,” Scott revealed.

Imagine having to manually retrain your AI Agent every time someone added a new business listing. “By introducing a self sort of help feature for businesses and events, it just wasn’t feasible to go in every time a new listing happened and click a button to sort of retrain it.”

But the real problems ran deeper:

The data wasn’t live and was completely mixed up. “If I were to ask a question about an event, and let’s say I’ve got an event page and then I’ve also got an article that mentions an event. It started getting confused between the two because there was no clear separation of the data. it was just sort of all thrown together, and that just added to sort of like a poor user experience.”

Version 2: The rebuild that changed everything

Instead of patching up the mess, Scott did something bold – he rebuilt most of the structure of the AI Agent. This time, he separated everything into clean databases: places, events, travel guides, and reviews each got their own home.

Here’s where it gets clever. When someone asks Mersey, “Is Coles open?” (that’s a supermarket chain for non-Aussies), here’s what happens:

1. Mersey figures out this is about a place, not an event
2. It calls a custom place tool
3. That tool checks the current date and time
4. It searches the places database for Coles
5. It comes back with “Is Cole currently open,” plus contact details and opening hours

If Coles were closed, Mersey would even tell you when it opens next. That’s the kind of practical help tourists actually need.

This rebuilding involved the creation of custom tools for Mersey that improved the functioning of the AI Agent.

The 100-hour reality check

This wasn’t a weekend project. “It was actually about over 100 hours of work gone into rebuilding the entire system, building the databases, building the website and platform. So it is a big time investment that you sort of need to big time investment if you’re going to go through that.”

The data structure revolution

The key insight that changed everything? Data quality.

“If you don’t get your databases correct and if you don’t have your data structured, I mean, it’s bad data, and if you’re putting bad data into an AI Agent, the output’s just going to be bad data. So, you really need to get your data sorted.”

Scott contrasted this with his previous setup: “When I was on the previous website, the data structure was terrible. It was a drag-and-drop page builder, so all the data was stored in the same sort of column in a table.”

The new system processes data before sending it to the AI Agent: “The data is pre-processed before being provided to tar. So I know that the data going to t is actually good quality data, which I couldn’t before.”

Self-service business portal

Scott built something clever – a business portal where companies can manage their own listings. “I’ve actually got a business portal. And this is a self-help business portal for businesses to actually submit their own events and sort of submit their own listings, and then they can edit listings as well.”

But he learned something about human nature: “I know from experience and especially even like big tourism data warehouses like we got in Australia, businesses just don’t want to update the data. They sign up and then they forget about it.”

The prompt engineering journey

Scott emphasized that building the AI Agent was an ongoing process: “Build a prompt with the knowledge that you’ll regularly need to fine-tune. I think I told you this before, six months ago, when I said that I regularly read the chat logs and I’m regularly changing that prompt so I can give it more sort of like detailed information.”

Real-world prompt challenges

One perfect example: “I had an issue with you, where do I where can I park with my caravan, and it would be goin,g telling them where to go to a caravan park versus an actual parking spot where they could park their caravan in the CBD. So, I had to provide more prompting.”

These kinds of misunderstandings only surface through real user interactions, which is why Scott stressed the importance of monitoring chat logs and continuously refining the prompts.

The personality factor that everyone forgets

He didn’t just throw together a random image: “I hired a designer to do my icon for my AI Agents. I had him design a lighthouse icon. I wanted to give it a bit more personality.”

The result? “I think a lot of people have sort of really liked that visit my site, that it’s a little lighthouse with a face.”

Measuring success (the manual way)

When asked about metrics and evaluation, Scott was refreshingly honest about his manual approach: “When it comes to actually working out how successful like version one all the it’s just as you read the chat logs but if I read something and I can’t I’m not getting the answer which I think the user wants and the questions are usually pretty direct even though there are some really weird ones that come through like that I don’t consider that successful.”

He also noted a gap in the platform: “There isn’t really a I don’t believe there’s way to view up votes and down votes for responses, which would be helpful.”

But Scott had ideas for scaling this: “I download hundreds of chat logs, and I use AI because I don’t want to read through all of them. I use I actually score the AI Agent in its accuracy and a certain number of conditions that I knew. So I think there is room to use AI to actually give a sort of generalized success score.”

The custom tools experience

Scott’s experience with Tars’ custom tools feature was smooth: “Once I had the once had the IP pass up in the actual schema generated in the format you needed. I mean, it was pretty easy just to paste it in, and yeah, and just connect it up, and it worked.”

For generating the API schema, Scott used a practical approach: “To get the schema, I used a bit of AI to work that out, and then I hadn’t used that schema format before. So I just for my API documentation, I made it into AI to actually get it to generate it.”

Lessons from the trenches

Scott’s key takeaways:

• Start simple, then build up complexity
• Clean, structured data beats fancy features
• Regular prompt fine-tuning is essential
• Give your Agent personality
• APIs work better than web scraping for complex data

The results

Today, Mersey provides actual, useful information. Ask about opening hours, and get the current status. Ask about weekend events, and get events actually happening this weekend. It is helping tourists navigate Australia’s largest city tourism site (eat your heart out, Sydney).

Scott’s journey shows that building truly useful AI Agents isn’t about having the fanciest AI – it’s about having the right data structure, clear purpose, and attention to user experience. Plus, a lighthouse with a face doesn’t hurt either.