"Unlock the potential of AI for your business: Explore black box, white box and grey box AI to find the best combination for your needs."
Background
In the ever-evolving landscape of artificial intelligence (AI) and machine learning (ML), understanding the nuances of various methodologies is key to leverage their potential in specific use cases. Such a nuance is the distinction between black box, white box, and grey box AI methods, each with its unique strengths and limitations.
Cracking open the black box and peeking inside
Black box methods give us answers, but they don't really tell us how they got those answers. It's like trying to understand how a magician performs a trick. These methods, including neural networks and deep learning, are great at solving complex problems but they don't easily show us their thought process. A recent example of this is Generative AI like ChatGPT and Google Bard. They can create human-like text; however, we can't always figure out exactly why they write what they do. These AI models are trained with lots of training data (often many millions of data sets) and often need a tremendous amount of computer power to work well. They can also be tricky to understand and may not be useful if we don't have enough data to train them, the data quality is not good enough, or if the data does not fit the requirements.
The clear logic of white box methods
Now, picture another AI box - one that's transparent. White box methods are like this. They let us see how they arrive at their answers, which can be really helpful. Examples of white box methods are semantic networks, knowledge graphs, or logic programming. Typically, these AI models need no training data and are normally created by modeling specific domain or expert knowledge as a set of logic rules or smart graph data structures. They are great for tasks where we want to apply specific knowledge, for example, understanding relationships between things or for logical reasoning. However, when the rules get complicated, complex, or incomplete, these methods might struggle to give good answers. They are also less useful for any type of deeper pattern recognition (e.g. image, audio, video, etc.).
The balancing act of grey box methods
Between black box and white box methods, we have grey box methods. At first sight, these methods combine the best of both worlds, but, unfortunately, they also have some limitations. Examples of grey box models are probabilistic methods and Bayesian networks. These can be trained using training cases similar to black box methods but can also be easily created with the help of domain knowledge, similar to white box methods. They are also fundamentally comprehensible to humans if they produce results. It's a little like blending two flavors together! Grey box methods try to make things clearer while also being good at generating results. Unfortunately, they can become quickly too big regarding memory and computer power if we're not careful.
Real-life examples: picking the right tool
Let's look at some real-world situations to understand when each method is useful:
Spotting cats in photos: When it comes to identifying cats in pictures, black box methods are like having a magical eye, especially if they are trained using sufficient data with the right quality. In comparison, attempting to utilize white box methods to explain what a cat looks like in strict logical terms would be really tough.
Checking legal contracts: Figuring out the coherence of legal contracts is a bit like solving a puzzle. Black box methods would need a lot of training data, but, with changing regulations for contracts, there may not be enough suitable training data available. Alternatively, white box methods could handle changing regulations better but might struggle with the basic understanding of the text and content. So, it might be useful to combine black-box, white-box and / or grey box methods to solve the task.
Thinking logically: When we want to create new knowledge from gathered facts based on reasoning, black box methods typically aren't very helpful as there’s no training data for the new knowledge available beforehand. In contrast, white box methods, often grounded in transparent logical principles and rational deduction, frequently prove more effective.
Driving safely: For self-driving cars, black box methods are ideal for object recognition in images and videos: they recognize e.g. other cars, cyclists and pedestrians as well as traffic signs. However, when it comes to understanding traffic rules, white-box methods that follow logical rule sets are much more useful. As a result, a combination of black box, white box and / or grey box methods must play a role to enable self-driving cars.
What you should and can do
Challenges and things to think about
Keep in mind that in certain scenarios AI decisions will need to be explained and understood, particularly for critical domains like robotic surgery in hospitals, assessment of creditworthiness in banks, or for weapon systems and military decisions. This is where relying solely on black box methods can get complicated at times.
Also, using training data owned by others could lead to legal issues (e.g. intellectual property (IP); particularly copyright issues). Similarly, using private or sensitive data for AI model training, as well as incorporating such data within inputs (prompts) or queries for certain Generative AI systems, could raise concerns under GDPR or CCPA regulations.
Using AI wisely to drive your business forward
If you're thinking about using AI, first figure out what you want to achieve and whether it really would improve your business. For example, is it making things faster, better, cheaper, or smarter? AI often isn't a magic wand, so be clear about what you want. Once you're ready, think about which method or mix of methods suits your needs. Ask the following questions: Do you have access to appropriate and sufficient training data? Do you need to know how the result is achieved? Is expert or domain knowledge available? What tasks need to be completed? Are there IP or GDPR / CCPA issues? Depending on the answers, sometimes you need the magic of black box methods; on other occasions, the clarity of white box methods or the flexibility of grey box models prove essential; but often a hybrid approach is best. If you are unsure, don't hesitate to seek expert advice.
Learn more
Join my webinars, video masterclasses, workshops, and coaching sessions to learn more about successful data-driven business models and scaling, metrics-based financial planning, investor readiness, and how to leverage technology and AI to move your business forward.
You can see an overview of my products and services (available in English and German) on my website: www.arndtschwaiger.com
I speak tech, product and business!
Let's scale digital and tech-based businesses together! 🚀
Join me on: LinkedIn, Instagram, Facebook and X
I am looking forward to your feedback and comments!
Thank you and best regards,
Arndt
