Add them to the allowed characters, but you'll need to escape some of them, such as -]/
var pattern = /^[a-zA-Z0-9!@#$%^&*()_+-=[]{};':"\|,.<>/?]*$/
That way you can remove any individual character you want to disallow.
Also, you want to include the start and end of string placemarkers ^ and $
Update:
As elclanrs understood (and the rest of us didn't, initially), the only special characters needing to be allowed in the pattern are &-._
/^[w&.-]+$/
[w] is the same as [a-zA-Z0-9_]
Though the dash doesn't need escaping when it's at the start or end of the list, I prefer to do it in case other characters are added. Additionally, the + means you need at least one of the listed characters. If zero is ok (ie an empty value), then replace it with a * instead:
/^[w&.-]*$/
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…