Mistral AI, launched in 2023, has been riding in the AI wave. If you follow the AI news, you must have heard about the massive fundings it has been securing. All of this got me wondering how well Mistral stands amongst its contenders. Could it replace my love for ChatGPT and Claude? Let’s see.
The Basics: Getting to Know MistralAI
Mistral AI, founded in Paris in April 2023 by former Google DeepMind researcher Arthur Mensch and ex-Meta AI scientists Guillaume Lample and Timothée Lacroix, is a European AI startup specializing in open-source generative AI models.
Their flagship products include Mistral Large, a general-purpose language model, and Le Chat, a chatbot designed for versatile text-based tasks.
Notably, Mistral AI has attracted significant investment from major tech firms like Microsoft, Nvidia, and Salesforce, underscoring its rapid ascent in the AI sector. The company’s commitment to open-source development and its European roots distinguish it from many U.S.-based AI enterprises.
Testing Mistral’s Capabilities (Prompt Test)
Mistral isn’t just a chatbot; it’s a product built for developers and tech companies, optimized for complex, technical queries. And while Le Chat is a standout, Mistral has more up its sleeve, promising a range of open-source AI tools tailored for deep customization.
For this article, we’ll be testing Mistral’s Le Chat using different prompts. We’re testing Le Chat based on how it performs under these diverse categories:
- Creativity and depth
- Technical accuracy and information
- Real-world applications
Let’s start with the first one.
Creativity and depth
I wanted to see how Mistral would perform when it is given a prompt that requires creative response along with some depth. And what better way to test that than asking it to write a fictional story? For some twist, I decided to limit the word count to 500 words.
Here’s a prompt I picked:
“Write a short story about a lost traveler who stumbles upon a hidden city in the desert. Describe the city and its people, and build suspense as the traveler realizes they may never be allowed to leave. Word limit is 500 words.”
If you read the story Le Chat generated, it’s pretty good. Especially the details. In the prompt, I’d told it to describe the city and its people and MistralAI did it in a good way. It even went into details about describing a woman, her eyes, and her tattoo. Overall, a pretty good result.
Then I decided to do some follow-up prompting and asked it to simplify the story a bit as it seems complex.
It simplified the words a lot, which is great. But the story is a bit boring now. So, I decided to follow-up with another prompt.
As we can see now, the results are much better. This is a great addition. And it also shows that Mistral isn’t just capable of providing the results but also does a great job at making the edits you may request.
Technical Accuracy and Information
Next, I decided to test how technically accurate information Mistral can provide. I went ahead with this prompt:
“Describe the impact of artificial intelligence in healthcare, specifically focusing on how machine learning improves patient diagnosis accuracy.”
I decided to ask it to provide me with sources for this information. And this is where Le Chat dwindled like most AI chatbots.
It could not give me the exact sources and just gave me a vague answer with all the sources that “may” have this information. This is a common theme amongst all AI chatbots.
We’ve also observed this in our AI chatbots comparison article.
I decided to take it up a notch and asked it a history related question. This is the prompt I used:
“Check these statements about World War II for accuracy and correct any errors you find:
- Adolf Hitler died on April 30, 1945 in Berlin, but his body was taken to Moscow
- The first atomic bomb was dropped on Hiroshima on August 6, 1945, followed by Nagasaki on August 8, 1945
- Germany officially surrendered on May 7, 1945, in Paris”
Here’s what I got:
Mistral identified all the errors and ended up correcting them. This means the AI is smart and reliable. But I still took this information with a grain of salt and verified it by Googling it all.
I decided to test technical accuracy with some sports records from the past.
Here’s the prompt I used:
“Review the following sports trivia:
- Serena Williams holds the record for the most Grand Slam singles titles in tennis history with 30 titles.
- Usain Bolt set the current men’s 100m world record at 9.68 seconds during the 2012 Olympics.
- The Brazilian national soccer team has won four FIFA World Cup titles, the last one in 1998.
Identify any inaccuracies in the records or statistics.”
I verified all these from the internet and yet again Mistral was correct. It did make a mistake by not mentioning that the 24 titles held by Margaret Court are the single. In total she has won 64 titles which includes doubles and mixed doubles.
Real-World Applications
Next up, I decided to test Le Chat with some real-world applications. My aim here was to see how much Mistral can simplify things for me.
I decided to go ahead with the following prompt:
“Summarize the main takeaways from an article from this article”
I tried pasting the article link but Mistral cannot read web links (ChatGPT and Gemini can do that). So, I pasted the article’s content into the chat. Here’s what I got:
While the summary could have been shorter (I can always ask Mistral to do that), it did a good job. It picked the key headings and summarized what was written in those. This is great when I do not want to read the whole thing.
The, I asked it to generate three LinkedIn posts, these are the post it gave me
It picked the three main themes and created a post out of each. However, the problem here is that I would not post these on my LinkedIn account. These read too AI-generated to be posted for an audience.
The same goes for Post 2. If you’ve read content on LinkedIn, no one writes like this. This needs some “setting stage” by talking about how I’ve been experimenting with some AI-detectors and here are my two cents about some of them.
Post 3 feels somewhat better than the two above. Although I still need to add the touch of my voice and style of writing into this.
Overall, I feel, Mistral gave me a great place to start. I can pick these content ideas and write the posts myself. Or I could do some manual edits and make the AI-generated post my own.
Other Capabilities of Mistral: Pixtral & Coding
Mistral has a model called Pixtral which processes both text and images. Similar to how you can upload images to ChatGPT, Gemini and Perplexity, you can do it with Pixtral. This AI feature is called “Vision capabilitiy”.
Pixtral
I decided to try Pixtral with the following prompt:
“Describe the content of this image in detail, including colors, objects, and any activities occurring.”
Here are the results:
Mistral went into detail describing the backpackers and then the scenery. I really like how it even explained details like the sky and the angle from which the image was taken. Overall, great image analysis.
Now let’s test some image captioning capabilities:
“Generate a creative caption for this image.”
Here are the results:
The caption was a little too wordy but with the help of follow-up prompting I could get a simpler caption.
Lastly, I wanted to see if Mistral’s Le Chat could extract content from a picture. I decided to use the FreshVanRoot landing page.
Here is the result:
Yet again, Mistral was able to extract just the right amount of content about the image. Honestly, this is quite impressive. With the help of more follow-up prompting, I can get some great results which can be used as a final product.
Mistral for Coding
Can Mistral code? I decided to try creating a custom social media app with it.
The app project turned out fine. Within 30 minutes I got an app created that I can use to create images for my LinkedIn profile.
Other Newer Capabilities with Mistral
To keep up with its contemporaries, Mistral has launched three new features: Image generation, Canvas and Web Search. Here’s how they work:
Image Generation
Just like ChatGPT’s Dall-E or many other AI-powered image generation tools, you can type in details of the image you want and Mistral will generate it for you.
This is the image Mistral gave me when I asked for a woman working in a corporate office in New York. The image is quite good. The quality is great as well. However, what’s missing is the New York element. But, with follow-up prompting, you can get that too.
Canvas
Canvas is similar to the Canvas offered by ChatGPT, which is quite impressive. For writers and content creators, this reduces the context switching. I do not need to keep switching tabs to write a piece.
One thing worth noting is that it does have features like reading level, final touches, or adding emojis. I cannot also pick a part of the text and specifically chat about it.
Web Search
Mistral can do web search as well. For example, I asked about Rolf Mistelbacher. It gave me the entire information about myself with sources listed in the footnotes. Web search is also available in ChatGPT, you can read our article on how it compares to Google.
I decided to ask something about the history and here’s what I got.
Ironically, there were no sources listed for this response.
Where Mistral Falls Short
Mistral brings a lot to the table with its AI capabilities, but there are a few areas where it could use some improvement, especially when compared to other tools.
One of the main drawbacks is the user interface. It feels basic, and if you’re accustomed to more sophisticated tools, this might be noticeable. Features like conversation grouping, adding artifacts, or having a visual canvas are missing. This makes organizing and navigating content less intuitive than what you’d get with other platforms.
Here are a few more limitations:
- No Third-Party App Integration: You can’t use or integrate third-party apps or create custom GPTs, limiting the customization and flexibility that some users need.
- Lack of Advanced Collaboration Tools: Unlike platforms that allow real-time co-editing, task assignments, or shared workspaces, Mistral doesn’t support those kinds of team-based interactions.
Verdict: Is Mistral a Real Competitor?
When comparing Mistral to other AI chat assistants like ChatGPT, Gemini, and Perplexity, I believe it stands as a strong contender. While Mistral does have some gaps in its capabilities, it still delivers impressive results overall.
For instance, ChatGPT currently outperforms Mistral when it comes to image-based prompting. However, considering the progress Mistral has made so far, it’s reasonable to expect that its Pixtral feature will continue to evolve and improve. That said, ChatGPT does have a slight edge in response quality.
Gemini’s seamless integration with Google Workspace is another point that’s hard to overlook. For those who rely on Google’s suite of tools daily, this makes Gemini a standout option.
Perplexity, on the other hand, brings unique features to the table, like its focus mode and video generation—features that Mistral hasn’t yet introduced.
In summary, while Mistral delivers strong, high-quality responses, it still has room to grow when it comes to feature offerings.