especially since Matt spent a lot of time with Elena growing up, but there's no real indication that Vicki was there as well.

It doesn't make much sense to me that Vicki and Jeremy didn't grow up as best friends, too, the way Matt and Elena did. It can probably be handwaved that they just weren't in the same grade at school (I think) the way their siblings were, but I thought Mr. and Mrs. Gilbert were friendly with Kelly, which would have meshed the two families regardless of minor age differences.