With B establishing an overlap to windward of A, A has full luffing rights and is under no obligation to sail her proper course.  

There is no rule requiring Boat A to sail a proper course.  She may luff Boat B to head to wind.  Boat A will have to give Boat B room to take down her spinnaker once she can no longer carry it before changing course to luff her higher.  If Boat B does not begin to take her spinnaker down when Boat A gives her that room, Boat A may resume luffing Boat B.

During the entire situation Boat B is required to keep clear of Boat A, first by rule 12 and, once overlapped, by rule 11.  Boat A is required to give Boat B room to keep clear when she changes course by rule 16.

Nick … to add a different angle to what Sue and John C have said (agreeing with both), the 2 questions I always ask someone when they ask about “proper course” in a scenario are …

Proper Course is only used in Rules: Mark-Room, 17, 18.1(b), 18.2(c)2, 18.4, 23.2.  So if none of those rules apply, then proper course has no impact on the situation. As aptly pointed out by both Sue and John … none of the above rules apply to your scenario.

#2 How is PC used?
Let’s say that a rule that uses the PC term did apply. How is PC used in that rule?. Each rule utilizes the term differently. For instance..

But as John C points out .. in all those uses and applications of “proper course”, NONE of them require a boat to sail a proper course. As he says above, “There is no rule requiring Boat A to sail a proper course.”

A point of clarity regarding John C's assertion:  "If Boat B does not begin to take her spinnaker down when Boat A gives her that room, Boat A may resume luffing Boat B."
This may be true, to a point, but it can become an infringement of RRS 14 and 16 for A.
A better action may be to protest B for failure to keep clear of a leeward boat under RRS 11 and resume sailing to the mark.

I am interestrd in John C's comment "Boat A will have to give Boat B room to take down her spinnaker once she can no longer carry it before changing course to luff her higher."

I interpret this to mean that A must stop luffing B once B needs to drop her spinaker. As long as A is giving B room to change course in response to her luff,  what Rule(s) prevent A from continuing with her luff?

Julian - this comes from the definition of room. The definition of room is : The space a boat needs in the existing conditions, including space to comply with her obligations under the rules of Part 2 and rule 31, while maneuvering promptly in a seamanlike way.

The key phrase is 'in a seamanlike way'.  It is not considered seamanlike like to use a spinnaker on a point of sail where it cannot be filled, i.e. is luffing.  So, once the leeward boat luffs to the point where the windward boat can no longer keep her spinnaker filled, the seamanlike thing for the windward boat to do is take the spinnaker down.  However, the room that the leeward boat has to give the windward boat also includes an aspect of time, i.e. if the windward boat chooses to continue to sail with her spinnaker luffing despite having the opportunity to douse her spinnaker and she chooses not to, then the leeward boat can then resume luffing.  If the windward boat chooses to douse her spinnaker, then the leeward boat has to give her the room required to perform the maneuver before resuming her luff.  Match Racing Call G2 explains this.

MR CALL G2
Rule 16.1 Changing Course
Definitions Room
Question 1
Blue is sailing to leeward of Yellow. Both boats have spinnakers set. Blue luffs
Yellow. What are the obligations on Blue and Yellow?
Answer 1
Yellow is required to keep clear and Blue is required to give Yellow room to respond
to her luff. While Yellow is able to keep the majority of her spinnaker filled and
drawing she is not entitled to additional room to drop her spinnaker.
If Blue continues her luff so that Yellow, in order to keep clear, has to luff beyond
the point where she can keep her spinnaker filled and drawing, then Blue must give
Yellow room to drop her spinnaker.
Whether or not Yellow drops her spinnaker at that time, she is still required to keep
clear. If she does not drop her spinnaker at that time, then Blue is not required to
give her additional room to drop the spinnaker later in that incident.
Question 2
If Yellow does not drop her spinnaker and passes head to wind to keep clear, is Blue
giving Yellow room to keep clear in a seamanlike way?
Answer 2
Yes. Blue may continue to luff as long as she has previously given Yellow room to
drop her spinnaker.
Question 3
Does either answer change if Blue does not have a spinnaker set, or has a different
type of spinnaker set?
Answer 3
No.

Julian re: room to take spinnaker down (old thread)

Here is an entire thread dedicated to that subject from 2019

https://www.racingrulesofsailing.org/posts/374-match-race-call-g2-giving-room-to-drop-a-spinnaker

John and Angelo - thanks. Very interesting reading.