Process the refund making sure to deduct all fees the customer is responsible for.

If the package was ‘refused’ by the buyer — and has been returned to you in the same packaging as you shipped the book — you, IMHO, should make the buyer whole.
The buyer has already paid you for the postage to get to them — though refused — the buyer has nothing to be debited for. The buyer is not responsible to pay return shipping on a refused book. Refused/Returns on Media Mail are one of the few types of returns that USPS requires to obtain postage. You are out the return postage — the buyer is not responsible to pay this.
CJ

I would process the refund for the item amount only, w/o shipping charges. Make a note of this in the space for message to customer. Choose Refused Delivery as the reason for refund. I would not charge a restocking fee under these circumstances.

I would check the USPS website to be certain the refused delivery was a refused delivery.

Tracking on Amazon, often calls other reasons for non-delivery refused delivery.

Many sellers would follow @Aardvark_Book_Depot advice. Others will also charge the restocking fee. I would want this issue resolved immediately, which means refunding in full. Not worth the potential effort required if the customer wishes to dispute this.

Some sellers watch every penny, others throw money at making problems disappear. Neither group understands the other.

Thank you everyone. I have refunded the money in full.

I had a similar situation where a customer refused a package, filed an A to Z, got the A to Z granted, and left a negative seller feedback all in the same day.

I suspect it was a competitor. Nevertheless, the shipping was $55. What do I need to file with Amazon to get that back? Amazon didn’t give me an opportunity to respond to the A to Z before it was granted.