Same as for Koh Jum the boat starts from Leam Kruad pier, see timetable below. Since we already were on Koh Jum, you need to take the boat back to Leam Kruad pier and change boat to reach Sriboya.
Another alternative is to rent a Longtail boat and go directly from Koh Jum to Sriboya. Since we were 10 people it was more economic to rent the Longtail, it cost us 1500thb and saved a lot of time.
We rented bungalows at Siboya Bungalows which was a good decision, very relaxed area. Clean and well-maintained resort. Lots of hammocks to take a nap in and also here a good kitchen and bar.
Compared to Koh Jum there are even less to do on Sriboya, just time to chill out.
We had a great time and will be back.
You can go to Koh Jum by boat from Leam Kruad pier, the ticket costs 100thb one way and the boat leaves about every 1,5 hours (see schedule in pics). From Krabi town you can catch a songthaew to Leam Kruad pier, but you need make one change on the way.
If you booked a room already, I suggest calling the resort and ask if they can pick you up from Mu Tu pier on Koh Jum.
We stayed at Jungle Hill bungalows which offered affordable bungalows by the sea, highly recommended. Good kitchen as well.
What to do on Koh Jum? Not much, just relax. Rent a motorbike from the resort and explore the island. Swim a lot, the water is clear, and you have plenty of beaches to choose from.