Recently, I've been hearing a lot about models that perform tabular prediction using ICL (In-Context Learning). In addition to Google TabFM, which I introduced last time, several startups have also released ICL models. Therefore, I would like to compare them with LightGBM, a machine learning model I frequently use, to see where their strengths lie. Let's dive right into the details.
1. Applying ICL (In-Context Learning) to Tabular Data
The selling point of ICL models is that they can be used straight out of the box without training. The chart below should make this specifically easier to understand.
Applying ICL (In-Context Learning) to Tabular Data
First, regarding the input data, the context is key. This involves gathering data with already known outcomes and showing it to the model beforehand to prepare it for prediction. Next, the samples (queries) for the actual prediction are input, and the prediction is executed. In reality, as shown in the diagram, the context and the samples (queries) are combined and input into the ICL model as one large prompt. Since we will be predicting approval or rejection based on loan application data this time (3), it will be a binary classification prediction task. Now, let's actually experiment with some data.
2. Comparison between ICL Models and LightGBM
For this experiment, I used a model called TabICLv2 (1). It is lightweight, easy to use, and its availability under the BSD 3-Clause License is also appealing. I compared it with LightGBM (2), a gradient boosting machine learning model. I have been using LightGBM as my first-choice model for over five years. I'm excited to see how close TabICLv2 can get to this LightGBM. By the way, the hardware used this time is a MacBook Air M4 with 24GB of RAM. The results are as follows:
TabICLv2 vs LightGBM
The most noticeable thing is the high accuracy of TabICLv2 with a training dataset of only 100 samples. While LightGBM requires model training before prediction, with 1,000 samples or fewer, the training data was simply too small, causing it to fall behind TabICLv2. The high accuracy with a small number of samples is due to the fact that TabICLv2 is a foundation model and has been thoroughly pre-trained. Because it possesses the necessary information for prediction from the start, it can predict without any issues even with a small number of samples. It's amazing!
TabICLv2 vs LightGBM
However, in terms of cost (computation time), LightGBM has the advantage. When performing ICL (In-Context Learning) with TabICLv2, you end up inputting a large amount of context every time you make a prediction. If the context is around 1,000 samples, there is no particular problem in practice, but at 5,000 samples, the computation time becomes noticeable. Also, this time, the number of prediction samples is fixed at 1,000. In actual business operations, it often exceeds 10,000, so it is presumed that TabICLv2 would be difficult to use in cases requiring real-time processing.
3. When Should You Use an ICL Model?
Based on the above experimental results, I recommend trying out ICL models in the following cases:
When you want to execute predictions with a small amount of data on hand The situation of "not being able to do anything until data accumulates" is often called the cold start problem. Immediately after launching a new product in finance, naturally, there is no data. In such cases, managing credit risk used to be difficult, but it seems ICL models can resolve this issue.
When there are no data analysis experts Training models like LightGBM still requires specialized knowledge. If you do not have such talent in-house, an ICL model, which completely eliminates the training process, becomes a reliable asset.
Conversely, if you have abundant data and data scientists in-house, LightGBM seems to have the upper hand. After all, having the right tool for the right job is important.
What did you think? Various new ICL-type models are being released, and I'm looking forward to future developments. At Toshi Stats, we plan to continue exploring the application of In-Context Learning (ICL) to tabular data. Stay tuned!
You can enjoy our video news “ToshiStats AI Weekly Review” from this link, too!
1) TabICLv2: A better, faster, scalable, and open tabular foundation model, Jingang Qu, David Holzmüller, Gaël Varoquaux, Marine Le Morvan, Feb 11, 2026
2) LightGBM, Microsoft Corporation
3) Loan Approval Prediction, kaggle, Oct 1, 2024
Copyright © 2026 ToshiStats Co., Ltd. All right reserved.
Notice: This is for educational purpose only. ToshiStats Co., Ltd. and I do not accept any responsibility or liability for loss or damage occasioned to any person or property through using materials, instructions, methods, algorithms or ideas contained herein, or acting or refraining from acting as a result of such use. ToshiStats Co., Ltd. and I expressly disclaim all implied warranties, including merchantability or fitness for any particular purpose. There will be no duty on ToshiStats Co., Ltd. and me to correct any errors or defects in the report, the codes and the software.