My opinion on those four universes being much stronger has two parts to it:
1. Goku and Vegeta are not necessarily weaker than the strongest of those universes, at least not weaker than they are compared to Beerus/Whis. (There are theories that Goku may already be stronger than Beerus, but I don't believe them. I do think Beerus is grooming Goku to take his place as God of Destruction, however) However, considering that the other fighters in Universe 7 are dwarfed by Goku and Vegeta in terms of power, it would make sense that the average power level would be lower. Most of the other fighters are chumps in terms of power level.
2. That this signals that there is going to be something important involving those universes at some point, and likely some point soon (after the tournament? Perhaps during the tournament we see some of those fighters?) It makes sense for Goku and Vegeta to have something to aspire to that's not in Universe 6, 7, or 9. In all likelihood, we're going to see some kind of universe hopping at some point. Vegeta wants to visit the Saiyans in universe 6, and Goku will want to seek out those fighters to test his mettle.