No, the issue is that the throughput is so low with these very small signal strengths that transferring a meaningful amount of data will result in large delays on that side as well. Speed of light isn't the only thing causing measurable latency.