When Google announced Gemini, I was excited to try it because it is a multi-modal model that works with text, images, code, audio, and video. Unlike ChatGPT, it can also access the internet and provide real-time information. However, after using it for over six months, I am moving back to ChatGPT.
On paper, the latest Gemini 1.5 Pro model outperforms OpenAI’s new ChatGPT 4.0. First, it supports double the tokens for the maximum sequence length compared to ChatGPT, which means it can understand the context of prompts much better. It also supports up to a 2 million token context window, allowing it to keep previous chats and uploaded files in memory while giving a response.
Check this table for reference:
| Feature | Gemini 1.5 Pro (Google DeepMind) | ChatGPT 4o (OpenAI) |
|---|---|---|
| Parameters Trained On | 137B | 1.5B |
| Architecture | Transformer-XL | Standard transformer (details not public) |
| Training Data | Web text, books, code | Not specified |
| Max Sequence Length (Input) | 2048 tokens | 1024 tokens |
| Context Window | Up to 2 million tokens | Up to 128,000 tokens |
Note that these details are revealed by the respective companies. The actual experience of using them would be different for most people. Here’s my experience in reality:
Table of Contents
1. Gemini App Replaces Google Assistant on Android
When you install the Gemini app on Android, you are forced to use Gemini as your digital assistant, replacing the familiar Google Assistant. This change isn’t welcome, as Gemini lacks many features you use daily with Google Assistant.
For starters, I cannot control my smart home, custom commands to control apps no longer work, and even niche features like the ability to identify songs by humming are missing. Additionally, Gemini struggles to understand what I am saying and operates much slower compared to Google Assistant. That’s possible since Gemini is still “training itself” using my voice commands. Google recently added the ability to set reminders via Gemini, this improvement was long overdue.
To continue using Google Assistant, I have to forgo installing the Gemini app and rely on its web version. Instead of replacing Google Assistant, why can’t Google integrate Gemini’s abilities into Google Assistant, like Apple enhanced Siri? For now, I conclude that Gemini does not have a proper app on Android and has many downsides.
If Gemini replaces Google Assistant on your phone, you can switch back to Google Assistant from Gemini.
2. Gets the Math Wrong Most of the Time
Recently, I moved to a new room with three friends. We paid for expenses for the house and wanted to split them equally at the end. I gave Gemini details about the brokerage, advance, transport, and who paid how much, asking it to calculate who should pay whom and how much to split the expenses equally.
Here’s the prompt I have given to Gemini.
I recently moved to a new room with three of my friends, and we paid various expenses without any specific order. Here's the breakdown of our expenses:
1. Advance Payment
Vinay: ₹22,000
Shyam: ₹40,000
Me: ₹10,000
2. Brokerage Fee
Vinay: ₹2,500
Shyam: ₹0
Me: ₹17,500
3. Transportation
Each person: ₹500
4. Purchases
Vinay: ₹11,000
Shyam: ₹20,000
Me: ₹1,800
We want to split all these expenses equally among the four of us. Can you help us calculate who should pay whom and how much to ensure everyone ends up paying the same total amount?Despite providing all the details to Gemini, it completely messed up the calculations. It suggested that my friend and I had to pay the same amount to the third person, even though my friend had paid more initially. Additionally, it stated that Shyam would receive ₹18,900, but didn’t explain who should pay him. Even after several back-and-forth messages to correct its calculation, I couldn’t get an accurate answer.
I finally had to revert to ChatGPT to get the problem sorted. ChatGPT provided a clean and correct calculation with the same prompt. Not only was the answer accurate, but the explanation was also detailed and clear.
This isn’t just a one-time issue. We also tried asking math questions, like solving arithmetic operations and finding derivatives of functions, where Gemini failed most times when the questions got complicated. It’s not just math; even with riddles or anything requiring reasoning and logic, Gemini has a higher chance of failure. It feels like Gemini isn’t ready for the everyday math yet.
3. Inconsistent and Incomplete Code Generation
I don’t program daily, but I occasionally like to play with APIs or learn what’s new as an enthusiast. With my limited testing, I can say Gemini has grown much better than its initial Bard days. However, ChatGPT still outshines Gemini in proficiency.
For example, I gave a detailed prompt to both Gemini and ChatGPT to create an expense tracker app using HTML, CSS, and JavaScript:
Build a user-friendly expense tracker web application using HTML, CSS, and JavaScript. The app should allow users to:
Add expenses: Users can enter details like amount, category (optional), and date/time for each expense.
Edit expenses: Provide the ability to modify existing expense entries (amount, category, date/time).
Delete expenses: Users should be able to easily remove unwanted expense entries.
Display expenses: Clearly present a list of all recorded expenses, ideally including relevant details like amount, category, and date/time.
Consider visual elements: Incorporate basic CSS styling to enhance the user experience and make the app visually appealing.Both models skipped the editing expenses part, but Gemini didn’t even mention it. Additionally, to set it up and get the output, I had to manually fix many minor errors with Gemini’s result.
The major issue for me with Gemini is that it lacks context when generating multiple codes. For example, I asked it to generate a login page, which it did successfully. However, when I asked it to generate the server-side script to handle the login logic, it created code with new variables that did not match the rest of the code. While the generated code was technically correct, it left more work and potential errors for me to resolve, which wasn’t the case with ChatGPT for the most part.
4. Lack of Explanation and Proper Structure
While Gemini occasionally provides a detailed explanation, it often skips this crucial part and jumps straight to the answer. Whether I ask it to generate code, solve a math problem or riddle, or explain a topic, I frequently find myself needing to request further clarification because the initial answer isn’t clear or well-structured.
For example, this is the prompt I have given:
The ages of 10 students in a class are: 15, 16, 14, 15, 16, 15, 17, 16, 14, and 15. Find the mean age of the students.Here’s the response from Gemini:
And this is what I got from ChatGPT:
Even when Gemini provides a detailed step-by-step explanation, I’ve found that the structure of its results isn’t as polished as ChatGPT’s. In my experience, using ChatGPT often leads to a more cohesive and clearer presentation, although this preference may vary from person to person.
5. Image Recognition Isn’t on Par with ChatGPT
In my usage of both services, I’ve uploaded various types of images for different purposes—memes for explanations, screenshots of error messages for issue resolution, and photos of objects to gather details. From my experience, ChatGPT excels in understanding images and retrieving relevant information. Google does not leverage it’s Google Lens capabilities which often does a better job than Gemini itself when it comes to finding details about any object, animal, or plant.
Another significant issue with Gemini is its inability to access images or files uploaded in previous messages. For instance, I provided Gemini with a list of math problems in an image and asked for the solution to the first one, which it accurately provided. However, when I requested the solution for the second problem, Gemini seemed unaware of what I was referring to. Even when prompted to refer back to the image sent in the previous message, Gemini was unable to access it. I tested this with multiple images and files, and Gemini consistently could only analyze images uploaded within the same message.
Furthermore, there are instances where Gemini doesn’t acknowledge uploaded images at all. For example, when I uploaded an image of a smartphone and asked for its model number, Gemini responded with details about its own Gemini model version instead. This issue persisted across both the free and advanced versions of Gemini.
6. Other Minor Issues With Gemini
- Concising text: When researching or writing articles, I use Gemini to check grammar, fact-check, or find alternative words and sentences. However, I’ve noticed that Gemini often shortens the text even when I haven’t requested it. I constantly have to remind it not to condense the text unnecessarily.
- Forgets the context: Gemini often loses track of the main topic when I diverge into a sub-topic during chats. While this can happen with all large language models (LLMs), Gemini appears to experience it more frequently. Even when I attempt to remind it of the original subject, Gemini can sometimes remain clueless.
7. Missing Features on Gemini
Gemini lacks several features that are available on ChatGPT:
- Custom Instructions: This feature allows users to set default instructions for ChatGPT to operate according to their preferences. For instance, users can instruct ChatGPT to explain information as if addressing a 15-year-old,
- ChatGPT Memory: ChatGPT includes a memory feature where it retains details you mention about yourself or your preferences during a chat session. This memory allows ChatGPT to keep context in later interactions, ensuring more personalized responses based on previous conversations.
- GPTs: GPTs, or Generative Pre-trained Transformers, are customized versions of ChatGPT tailored for specific tasks with additional information and rules. For example, there are GPTs designed to recommend books, some are optimized for tasks like composing emails.
- Temporary Chat: If you don’t want ChatGPT to save any chat in its history or respond with info from its memory, then you can start a temporary chat. It’s like Incognito mode but for ChatGPT.
- Voice Chat: The ChatGPT app offers a voice chat option, allowing you to interact with ChatGPT using your voice. While Gemini can also function as a voice assistant on Android, ChatGPT’s voice chat feels more authentic with its human-like voice, natural pauses, and responses that don’t sound robotic.
Each of these features can be a major reason to prefer ChatGPT over Gemini, as they collectively provide a more comprehensive, user-friendly, and flexible experience.
Moving Forward
After my run-ins with Gemini for a long time, I will return to using ChatGPT as my primary go-to chatbot. I prefer using Perplexity or other Perplexity alternatives for research because it has real-time info and explains in simple bulleted points. However, I will still be using Gemini for select tasks and purposes.
