> The whole energy of the swarm will be used there
that sounds like a made up problem. the Dyson swarm isn't to collect energy to send to the home planet. it's to collect energy. where that is used is going to be wherever it's needed. mining the asteroids, local computing (the cloud is no longer just a computer on earth, it's the cloud of the swarm elements), powering interstellar trips remotely, etc. the only thing that needs to get to earth is the imports of goods and services.
that sounds like a made up problem. the Dyson swarm isn't to collect energy to send to the home planet. it's to collect energy. where that is used is going to be wherever it's needed. mining the asteroids, local computing (the cloud is no longer just a computer on earth, it's the cloud of the swarm elements), powering interstellar trips remotely, etc. the only thing that needs to get to earth is the imports of goods and services.