How Different Training Goals Change What a Model Actually Notices

I’ve been spending some time thinking about a simple question: how much does the training objective really shape what a neural network learns? Not just performance numbers, but what it actually “pays attention to.”

One idea that kept coming back to me is training a model to predict whether an image is rotated like figuring out if something is upside down instead of directly telling it what the object is. At first I thought, okay, that sounds like a toy problem. But the more I sat with it, the more interesting it felt.

If you think about it, to tell whether an image is upright, you need some basic understanding of the world. You need to notice where the ground usually is, how animals stand, which way faces point, or what looks “natural.” It’s not memorizing labels, it’s building a kind of visual common sense.

And that got me wondering about something deeper: if I take two identical networks and train them differently, do they end up seeing the same image in the same way? Or do they develop different instincts, almost like people who grow up learning different skills?

Recently, My Senior asked me two interesting questions around this idea, and I found myself thinking about them for quite a while. They sound simple at first, but the more I reflected on them, the more they revealed about how models actually learn not just technically, but conceptually.

Rotation learning vs classification: what feels different to me

When I imagine a model learning from rotations, it feels like it’s trying to understand the scene as a whole. It’s not obsessing over whether something is a cat or a horse. it’s asking, “Does this look right?” It probably spreads its attention across the image, picking up on posture, balance, and overall structure.

On the other hand, a model trained for classification feels more like it’s hunting for clues. It wants signals that separate categories. Maybe it locks onto ears, whiskers, or textures, anything that helps it say “this belongs here.” It’s a bit more narrow in focus because the task pushes it that way.

Neither is wrong. They’re just learning different habits.

If I show horses and cats, what might happen?

I like to picture this as two people looking at the same photo.

The rotation-focused model might be thinking: “Where’s the head? Are the legs pointing down? Does the scene look stable?” It’s trying to make sense of orientation, almost like checking whether the photo was flipped.

The classification-focused model might be thinking: “Those ears look pointed… the face is small… probably a cat.” It’s scanning for identity clues.

Same image, different questions so naturally, different attention.

A small analogy that helped me

It reminds me of learning to draw versus learning to identify species.

If you learn drawing, you spend time noticing proportions, angles, how things sit in space. You develop a broad visual sensitivity.

If you study species classification, you focus on distinguishing traits like patterns, markers, defining features.

Both involve looking carefully, but your mindset changes what you notice.

Why this feels important

The more I think about it, the more I realize that architecture alone doesn’t tell the full story. We often talk about model size or layers, but the objective quietly shapes the kind of understanding that emerges.

It’s almost like the training task acts as a lens. Change the lens, and the same model starts noticing different things.

This also makes me wonder how many behaviors we see in models like good or bad, are really reflections of the questions we asked during training.

Something I keep coming back to

What I find fascinating is that a model trained on something as simple as orientation can end up learning surprisingly rich patterns. It’s a reminder that learning doesn’t always need explicit instructions. Sometimes, giving the right kind of puzzle is enough.

And maybe that’s the bigger lesson: models don’t just learn from data, they learn from the perspective we impose through the task.

I’m still thinking about what this means for building systems that generalize well. It feels like choosing the objective is less about optimization and more about deciding what kind of “intuition” we want the model to develop.