TLDR:
Kumo Small (28M) solves inverse kinematics from 512 in-context examples, at 2.4 cm position RMSE.
Problem Formulation
Grab a robotics textbook12 and it’ll teach you how to compute a configuration of your robot so that it achieves a target, usually a specific pose of its end effectors. This is the inverse kinematics (IK) problem: a function from end-effector pose to joint angles. For our SO-101 arm, the input is the gripper pose and the output is five joint angles: shoulder pan, shoulder lift, elbow flex, wrist flex, and wrist roll. In motion planning, additional constraints can be added to ensure smooth, efficient trajectories between multiple target positions.
Even in the modern world where the answer to reaching SOTA performance is often to throw a ton of data at a Transformer model, learned policies tend to output target poses for end effectors, which then requires solving IK as a secondary step. This is odd, because IK is solved per embodiment, which requires a description of the robot’s exact physical structure for good performance, e.g. as a URDF file3.
Yet humans are clearly capable of adapting to their changing physical structure without ever explicitly describing it. As children grow into teenagers, they undergo massive changes to their physical structure. Aging changes muscle tension and joint health more mildly, and that shifts the IK solution too. Humans seem to adapt regardless, perhaps because they:
- operate in a closed loop, i.e. a clumsy teen that misses their cup will quickly recover by trying again with a new configuration
- continuously calibrate by performing simple actions, e.g. reaching to rub their eyes in the morning
My hypothesis:
Explicit descriptions of kinematic chains are not necessary for IK. The function from pose to joint angles can be inferred from a few in-context examples.
In essence we want a non-parametric model of the IK function, so the obvious candidate is a k-NN, where the keys are target poses and the values are joint angles. A k-NN benefits from kernel selection, setting scales, and feature pre-processing. Those choices are inductive biases, and getting them right takes a fair amount of analysis. With a plain Euclidean distance the model has none of them and needs a very high density of samples.
Fortunately, Prior-Data Fitted Networks (PFNs)4 address this by training a Transformer on a diverse synthetic dataset. The resulting priors make those choices automatically. The attention layers are, in a sense, the non-parametric model. Because they are stacked, adaptation (recognizing the data distribution and setting the kernel and its scales) is blended with interpolation (retrieving neighbors and merging their values). Together this sharply improves sample efficiency.
Experiment Setup
An SO-101 arm is labeled with four AprilTags on its gripper and one on the base. A camera is mounted overhead. For five minutes at 30Hz we teleoperate the arm to perform simple pick and place actions. From the AprilTags we compute a non-metric end-effector pose with the PnP algorithm and a virtual pinhole camera model. The result is a bank of examples of that function, each an end-effector pose paired with the joint angles that achieved it. These examples are used in-context to predict joint angles for new poses. The data is split into train, validation and test sets (5185, 1742 and 1754 rows, respectively), where the validation data is used in the grid searches and the final demo (video above) is generated from the test set.
Our metrics are:
- train-range-weighted MAE of the joint angles. Let $e_j$ be the mean absolute error of joint $j$ in degrees, and let $r_j$ be that joint’s training range, the gap between its 0.5 and 99.5 percentiles in the training set:
$$\mathrm{MAE}_w = \sum_j \frac{r_j}{\sum_k r_k}, e_j$$
- end-effector pose root mean square error (RMSE) as estimated using forward kinematics of the robot
Model size
We use the latest tabular foundation models from Nvidia: Kumo Tabular Small (28M params) and Large (215M params)5. We compare their regression variants. These are the best currently available PFNs with solid performance across a wide range of problems6.
Context size
The Kumo models use inducing-point attention over rows so their computation time grows linearly with the context size. This means there is a trade-off between time to compute and output quality. We run a sweep over sizes $K \in \{16, 32, 64, 128, 256, 512, 1024, 2048\}$.
Target pose representation
The standard representation of pose is a combination of 3D coordinates for position and a representation of orientation like a quaternion or rotation matrix. In addition to this, we also compare a marker tetrahedron, the four AprilTag positions, with the corners of a regular tetrahedron that follows the gripper pose. Both yield 12-dimensional representations. Although redundant, they were tested because 3D positions might elicit simpler kernels in the PFN than those necessary to process orientation representations.
Context-sampling strategy
Our training data consists of real trajectories from the SO-101 arm, which makes it highly redundant as consecutive steps are strongly correlated. Hence, it makes sense to cluster the data and sample from the clusters. In addition to random sampling from the training set, we tried L1 clustering over joint angles and L1 farthest-point sampling.
Results
The sweeps below are scored on the validation set. We select on train-range-weighted joint-angle MAE. Each ± is the standard deviation over three seeds only. For random sampling, a seed draws which K training rows are used, and those same three draws are shared across models and encodings. L1 clustering keeps the same rows for every seed, so its spread is the row order and Kumo’s fit. The k-NN baseline does not depend on a seed. The winning configuration is frozen and then run on the test set.
Baseline
The baseline is a k-NN on the regular-tetrahedron poses. It takes the joint angles of the nearest training row, or the mean of the three or five nearest, under Euclidean distance. Error falls with the search-pool size and then levels off. The best validation result is a 5-NN over 2,048 L1-clustered rows, at 18.37 degrees. The last point is all 5,185 training rows, and it does not improve on that.

Pose representation and model size
Under random contexts, the regular tetrahedron has the lowest error of the four encodings, and the four are close: every best setting lands between 6.2 and 6.8 degrees. Large reaches 6.16 degrees at K=1024. Small reaches 6.57 degrees at K=512. Raising Small to K=2048 does not improve its best encoding.
| Model | Encoding | Best K | Weighted MAE (deg) |
|---|---|---|---|
| Small | Quaternion | 1024 | 6.62 ± 0.46 |
| Small | Rotation matrix | 2048 | 6.75 ± 0.18 |
| Small | Marker tetrahedron | 512 | 6.76 ± 0.49 |
| Small | Regular tetrahedron | 512 | 6.57 ± 0.25 |
| Large | Quaternion | 512 | 6.28 ± 0.52 |
| Large | Rotation matrix | 1024 | 6.39 ± 0.19 |
| Large | Marker tetrahedron | 512 | 6.46 ± 0.64 |
| Large | Regular tetrahedron | 1024 | 6.16 ± 0.17 |
Context sampling
Fixing Small and the regular tetrahedron, we compare random rows, L1 farthest-point sampling, and L1 clustering. Clustering at K=512 is best, at 5.91 degrees, 9.9% below random sampling at the same K. Farthest-point sampling peaks earlier, at 6.37 degrees with K=256. Past each method’s best K, the error rises. Extra rows from these trajectories are strongly correlated, and they dilute the context. The bars are the standard deviation over the three seeds.

Clustering also puts Small ahead of the best Large result under random context, 5.91 against 6.16 degrees. We take Small to the test set.
Test
The frozen configuration is Kumo Small, the regular tetrahedron, K=512, and joint-space L1 clustering. The test trajectory is easier than validation, and the weighted MAE drops to 4.62 degrees. Forward kinematics puts the end-effector position RMSE at 2.4 cm.
| Metric | Test result |
|---|---|
| Weighted MAE | 4.62 ± 0.04 deg |
| FK position RMSE | 2.40 ± 0.07 cm |
| FK position median | 1.74 cm |
| FK position 95th percentile | 4.07 cm |
On the same test set, the best k-NN is still the 5-NN over 2,048 L1-clustered rows, at 14.93 degrees, 3.23 times Kumo’s weighted MAE.
Discussion
We used simple tasks like pick and place to generate our training data, which is realistic but suboptimal for covering the pose space thoroughly. A trajectory visiting poses sampled from a Sobol sequence7 and interpolated with splines (for smooth motion) would be much better, but requires having an existing IK solution to move the robot to those target poses. On the other hand, Sobol sampling in configuration space (joint angles) is dangerous and my simulations show it makes the robot arm swing wildly. There is likely a middle-ground to be found.
Since we use a non-parametric model, the context size provides a notion of effective combinatorial complexity of the IK problem. We find that even as few as 512 samples are enough to achieve decent performance. Of course, we would never use this approach in production, but it raises further questions, like whether the robot policy could be encoded in-context in its entirety as well.
A recent workshop paper8 studies this question and finds that a 13-neighbor k-NN policy achieves a 91.7% success rate on LIBERO using approximately 137k observations from demonstration trajectories. The clear next step is to replace k-NN with a tabular foundation model for supervised action prediction, using the same pretrained visual and language features. Comparing learning curves across different dataset sizes would help isolate how much the tabular foundation model’s inductive bias improves sample efficiency over nearest-neighbor retrieval.