Short answer, no, size does not matter in regards to frame size.
Alternatively, you should shoot the largest caliber that you can comfortably and accurately shoot for self defense in my opinion. Ballistics come into play at some point, but at what point is up to you and your application. Frame size is purely ergonomics and comfort as far as OC goes. Most people who carry a "larger" pistol while OCing usually only say it that way because they carry a smaller pistol for CC, so their OC weapon is "bigger" by default.
I OC a Sig P229 .40 S&W with 2 spare mags. I feel it's the perfect size for me, and the .40 is a very good performer ballistically and is one of few "high pressure" self defense rounds. I have large hands so the large grip and double stack mag are just what I need. It's basically worthless as far as accuracy past about 150 feet though, so it's definitely my "in town" carry. It is very practical for such an application, however. If you want a heavy slow bullet, go .45. If you want a smaller faster bullet go 9mm. If you want the best "in town" self defense round ever invented (IMO) go 10mm. These are all semi-auto suggestions. Revolvers are a different story.
For my "in the woods" carry, I'm in the process of "building" a stainless Sig 1911 converted to .460 Rowland, which is essentially a hand cannon. A .44 mag-type power plant in a semi-auto. It's impractical in the city, but I can kill any wild animal in North America with it, and if necessary, it's known to be accurate at 175+ yards and packs enough punch to seriously injure someone even wearing light body armor. Anything further than 175yds or if I have time to switch weapons on the fly, I utilize my rifle.