The battle against email spam is a constant challenge in today’s business world. Our journey in improving spam detection has taken us from traditional methods to the forefront of AI-driven solutions. In this article, we share the evolution of our approach to spam detection.
Our Challenge: Overcoming Spam in Salesforce Cases
First, let’s dive into the unique challenges we encountered with Microsoft 365 and Salesforce. We received emails through shared mailboxes like info@ in Microsoft 365, which were automatically forwarded to Salesforce as cases. This process created an unexpected challenge: a lack of effective anti-spam filters. Without filters in Microsoft 365 and Salesforce, unfiltered spam mails flooded our Salesforce cases, leading to inefficiency and an increased workload for our teams.
First Steps: Limited Protection with Traditional AI
Our initial attempts to tackle spam with traditional AI, such as SpamAssassin with Bayesian filters, offered limited protection. We were only able to filter about 25% of the spam, prompting us to look for further improvements.
ADA: A Leap Forward
Faced with these limitations, in 2022 we took our first steps towards a more advanced AI solution. We developed a custom model based on ADA, a predecessor of ChatGPT 3.5. This model significantly improved our spam detection, achieving an accuracy ratio of about 90%. A key advantage of this ADA-based model was that it could be gradually improved through fine-tuning, while operational costs remained surprisingly low.
Transition from ADA to GPT-3.5 Turbo
As ADA’s lifespan came to an end, we sought more advanced AI solutions and chose GPT-3.5 Turbo. Although this step was an advancement in technology, the standard version of GPT-3.5 Turbo did not meet our expectations, with only 60% accuracy in spam detection. This outcome highlighted the need for a more specialized approach.
Our Own GPT-3.5 Model: The Breakthrough
Taking matters into our own hands, we developed a custom GPT-3.5 model. With 10,000 self-collected ham and spam emails, we achieved an accuracy of 99%. However, further fine-tuning of this model presented challenges. Adding only spam caused the model to classify everything as spam, and vice versa with ham. Despite the theory that existing models can be further fine-tuned, this approach proved to be ineffective in practice.
A Large-Scale Solution: 99.7% Accuracy
Our solution? A large-scale approach. We trained a completely new model with 30,000 emails, in combination with SpamAssassin. This achieved an accuracy of 99.7%, proving that extensive datasets are key to successful AI.
Conclusion: The Pillar of Our Email Automation
Our journey in AI-driven spam detection not only underscores our ongoing innovation and adaptation but also how this technology forms the backbone of our overall email automation. By efficiently detecting and filtering spam as the first step, we prevent wasting valuable time and resources on processing unwanted emails. This ensures that our advanced AI can focus on answering legitimate customer inquiries.
