"By the time I got there they had finished" is correct. It is saying that
"when I eventually arrived, they had already finished the job.""At the time" is referring to a time period in the past.
"My friends went to the beach last month, but
at the time I was sick and couldn't go."
"
At the time of the dinosaurs the earth was very different."