It could be your posture. It could be your muscle/muscle tone (especially if you unintentionally favour one side when doing things like sit ups). It could be fat distribution. It could be all in your head.
No matter what, you're probably the only one that notices so I'd try not to worry too much and just continue working out.
One side of my ribcage is bigger than the other (the bones are a noticeably different shape if I run my fingers along them), but I have to suck my stomach in and point it out for it to be apparent to other people.