With her swift motion, his arm too jerks toward his sword pommel, but freezes there when she hesitates. He's looking quite intensely at her now, jaw clamped and nervous.
The woman didn't attack him, so there's no need to escalate, but that first impression could've been better.

"Who are you," he demands quietly.