Is that the gulf island ferry from Tsawwassen you are taking? I usually do Tsawwassen - Swartz Bay - Fulford Harbour. Usually I take the 10am from Tsawwassen, arrive at Swartz Bay at 12:30pm, then take the 1pm to Fulford Harbour and I will arrive at the resort just before 2pm.
Rod, take a look at the ferries to Long Harbour on Saltspring.
My itinerary:
Friday leave Tsawassen at 10:25 and get to Long Harbour at 11:50. (direct I think)
Monday Leave Long Harbour at 3:30 and get to Tsawassen at 6:00 (couple of stops I think)
I may extend the trip by one day if my schedule allows it.
Long Harbour is on the NorthEast side of the Island, not far from Ganges and St Marys Lake.
Let me know if you find a better way to do it. Maybe the Fulford/Swartz Bay route would allow a later return on Sunday?