Many of you might be struggling to decide which generative AI to choose, caught between high-performance but costly proprietary models (like the famous GPT and Opus) and cost-free open-weights models where it's uncertain just how many tasks they can handle. With Jensen Huang (Founder and CEO of NVIDIA) publishing a report on X titled "Open Weights and American AI Leadership" (1), the momentum supporting open-weights models has grown. Therefore, this time, I would like to actually run both models on a customer churn prediction task and compare which one is better.

1. Gemini 3.6 Flash and Gemma 4

Both models are from Google. Gemini 3.6 Flash (2) is a proprietary model that strikes a great balance between accuracy, speed, and cost, making it the perfect baseline for comparison. Gemma 4 (3) is an open-weights model, divided into five types based on its size. This time, keeping in mind running it on a PC without a GPU like a standard Windows PC, I would like to focus the comparison on the smallest Gemma 4 E2B QAT. This generative AI can even be deployed on small devices like mobile phones, and I am excited to see how well this model can hold its own against a large proprietary model. By the way, the PC I used is a MacBook Air M4-24GB. Let's get started!

2. Generating Special Offer Proposal Texts for Customers

For the first experiment, I would like to actually try proposing preferential treatment to customers with a high probability of churning before they leave. The keys are: 1. Identifying customers who are likely to churn, and 2. Whether we can execute the notification of the preferential treatment offer in a timely manner. Here, we will leave 1 to a powerful machine learning library called LightGBM, and for 2, we will use generative AI to generate both the explanation of the analysis results and the notification letter to compare them. First, let's generate the analysis explanation text using Gemini 3.6 Flash as a baseline. The result is below. It is carefully explained in natural English.

‍ ‍ Analysis Explanation by Gemini 3.6 Flash

Next, let's try it with Gemma 4 E2B QAT. The results are as follows. This is also written in natural English. It's wonderful. Although it is the smallest model, it functions perfectly well. It looks like it could be immediately used in actual business operations. It is incredibly appreciated that a generative AI of this level can be used for free.

‍ ‍ Analysis Explanation by Gemma 4 E2B QAT

Now, I would like to create a notification letter for preferential treatment to prevent customers from churning. First up is Gemini 3.6 Flash, and as expected, it's perfect.

‍ ‍ Preferential Treatment Notification Letter by Gemini 3.6 Flash

Gemma 4 E2B QAT doesn't fall behind either. This is also written in natural Japanese and has sufficient content. With this, I think it has reached a quality that can be sent to customers. I am a bit moved that even an open-weights model can do this much.

‍ ‍ Preferential Treatment Notification Letter by Gemma 4 E2B QAT

I have summarized the results here. I have added two open-weights models with slightly larger parameters here. The time taken for computation inevitably gets longer as the parameters increase. By the way, even if you use these models, naturally, there are no usage fees. That is the wonderful thing about open-weights models. Gemini 3.6 Flash is the only proprietary model, and it boasts extremely fast speeds. It fully demonstrated its performance here as well, recording the second-fastest speed. This is a point worth noting. However, it does incur a cost, so budget management is essential when a large amount of processing is required. This cost is calculated based on generating the three types of texts.

3. On-Demand Data Analysis

This time, I would like to provide data to the generative AI and execute data analysis. I will visualize what percentage of the total customers are likely to churn. This is information that marketing personnel would definitely want. Normally, this is a task performed by data scientists who can write Python, but this time I want to challenge how far we can take data analysis by giving instructions via prompts and having the generative AI generate the Python code. First, let's try it with Gemini 3.6 Flash. As expected, it turned out perfectly.

‍　　‍　　　　　Visualization of Customer Distribution by Gemini 3.6 Flash

Next, I wanted to show the results for gemma4:E2b-it-qat, but unfortunately, it did not succeed after 3 attempts. It seems to have been a somewhat difficult task for a small generative AI. This time, all the prompt instructions were fixed and unchanged, but I think there is a possibility that even a small generative AI could succeed with some adjustments. The larger gemma4:E4b-it-qat worked out well, but gemma4:12b-it-qat failed as shown below. Code generation tasks might be quite a heavy burden for gemma4.

‍　　‍　　　　　　　Visualization of Customer Distribution by gemma4:E4b-it-qat

‍　　‍　　　　　　　　　Visualization of Customer Distribution by gemma4:12b-it-qat

What did you think? As we found out from this experiment, the cases where you would want to prioritize using an open-weights model are:

When handling confidential information and you want to complete the processing entirely on a local PC.
When budget management is strict and you cannot bear the costs.
When the task is centered around text generation.

Code generation tasks might reach a practical level if the prompts are adjusted. We at Toshi Stats will continue to take on these challenges. Stay tuned!

You can enjoy our video news “ToshiStats AI Weekly Review” from this link, too!

1) Open Weights and American AI Leadership, Jensen Huang (Founder and CEO of NVIDIA), July 24, 2026
2) Gemini 3.6 Flash Best for token efficiency in coding, knowledge work, and multimodal tasks, Google DeepMind
3) Gemma 4 QAT models: Optimizing model compression for mobile and laptop efficiency, Google, Jun 05, 2026

Notice: This is for educational purpose only. ToshiStats Co., Ltd. and I do not accept any responsibility or liability for loss or damage occasioned to any person or property through using materials, instructions, methods, algorithms or ideas contained herein, or acting or refraining from acting as a result of such use. ToshiStats Co., Ltd. and I expressly disclaim all implied warranties, including merchantability or fitness for any particular purpose. There will be no duty on ToshiStats Co., Ltd. and me to correct any errors or defects in the report, the codes and the software.